Redux: Synchronizing Firefox Through Dropbox

For the most part, the Dropbox-enabled synchronization of Firefox that I wrote about a couple of months ago has been reasonably solid. A few hiccups and a few annoyances, but nothing serious or even significant enough to warrant a change of approach. This morning, though, that changed and I needed to re-evaluate my synchronization strategy.

Yesterday.

Until today, I’ve been running something of a kitchen sink mechanism. You can read the full post for the details, but I was sharing my entire Firefox profile across multiple systems and platforms using Dropbox activity. Bookmarks, extensions, cookies, history and everything else. The transparent and immediate way that Dropbox works made this possible.

Over the last couple of months of using this method across one Mac client, one Linux client and two lesser used Windows clients, I’ve been bothered by several side effects:

  • Sharing profiles is chatty. Really chatty. I noticed this immediately and even mentioned it in my original post, but damn. Sharing an entire profile means a lot of Dropbox activity. Turning off system tray notifications in the preferences helped, but I kid you not when I tell you that the little blue animated icon overlay never stopped spinning. Unless Firefox wasn’t running at all, that thing was spinning like a madman.
  • When I move from machine to machine, I’m not as diligent as I should be about quitting Firefox on the machine I’m leaving. Because of this, I assume, I ended up with a lot conflicted files in my Dropbox-mounted, shared profile directory after a while. This isn’t really a problem, as far as I know, but I hate clutter.
  • Virtually every time I launched Firefox, I got the prompt that tells me my last session quit unexpectedly and asks whether I want to start a new session or restore my last. Because of the difference between how Mac and Windows/Linux order OK/Cancel buttons (even when the buttons don’t actually say OK or Cancel), I’d often find myself hitting the wrong one. Remember, I work in all platforms.

All of these actions by both applications, of course, are absolutely correct. Both Dropbox and Firefox were behaving exactly as expected, but the necessary side effects of the architecture I had concocted were still annoying.

Today.

This morning things got a bit more dire. Firefox simply wouldn’t launch while I was at work on my Linux machine. No matter how I tried – command line, Applications menu, Gnome Do, etc. – Firefox just wouldn’t start. There was no error or indication of an issue, Firefox just, well, did nothing at all. When I launched Firefox into the Profile Manager, though, I was able to select the default profile rather than my Dropbox-ified profile and run Firefox happily.

Knowing that the problem was in my shared profile created a level of severity that I needed to address.

Fixed.

Now that simple annoyance had turned to need, I set about defining a less brutish synchronization mechanism. What I really want synchronized across all of my machines – the profile elements that are most needed and yet the biggest pain to synchronize manually – are my bookmarks, my extensions and my Greasemonkey scripts so I decided to apply a more targeted solution.

Leaving my shared profile directory in place, I created a new Dropbox directory outside of my shared profile directory. To this directory, I copied what I wanted to synchronize from my Dropbox-shared profile. Then, in my default profile – the profile that is local to each machine rather than shared en masse via Dropbox – I created symbolic links to the bits I just copied.

Here’s what it looks like from the shell:


$ mkdir ~/Dropbox/Application Support/firefox/profile-share
$ cd ~/Dropbox/Application Support/firefox/profile-share
$ cp ~/Dropbox/Application Support/firefox/profiles/wg3x0vhj.dropbox/bookmarks.html .
$ cp ~/Dropbox/Application Support/firefox/profiles/wg3x0vhj.dropbox/places.sqlite .
$ cp -r ~/Dropbox/Application Support/firefox/profiles/wg3x0vhj.dropbox/extensions/ .
$ cp -r ~/Dropbox/Application Support/firefox/profiles/wg3x0vhj.dropbox/gm_scripts/ .
$ cd ~/.mozilla/firefox/erbbyfam.default
$ mv bookmarks.html bookmarks.html.orig
$ ln -s ~/Dropbox/Application Support/firefox/profile-share/bookmarks.html bookmarks.html
$ mv places.sqlite places.sqlite.orig
$ ln -s ~/Dropbox/Application Support/firefox/profile-share/places.sqlite places.sqlite
$ mv extensions extensions.orig
$ ln -s ~/Dropbox/Application Support/firefox/profile-share/extensions extensions
$ mv gm_scripts gm_scripts.orig
$ ln -s ~/Dropbox/Application Support/firefox/profile-share/gm_scripts gm_scripts

Details.

Although I thought I knew which files and directories I needed to create my targeted solution, I spent a little time verifying just to be sure. Here are the guts of what I wanted synchronized:

  • In Firefox 3, bookmarks are stored in profile root>/places.sqlite.
  • My application launchers (Quicksilver, Gnome Do & Launchy) key off of profile root>/bookmarks.html, so I needed to synchronize that, too.
  • Extensions are stored in profile root>/extensions/.
  • Greasemonkey scripts are stored in profile root>/gm_scripts/.

The nitty-gritty is pretty readable in the shell syntax above, but here’s the high level:

  1. I created a new directory so I could keep my targeted synchronizations independent of my complete profile synchronization.
  2. From my shared profile, I copied each resource into my new directory.
  3. In my local, default Firefox profile, I renamed its copy of each resource by appending .orig.
  4. In that same local profile, I created a symbolic link for each resource to the shared version in my new directory.
  5. I restarted Firefox and my default profile looked like my old profile in all of the ways that matter to me.

Now.

Hopefully this targeted solution will be much less flaky. Of course, my browsing history, form input history, cookies, etc. won’t be synchronized, but I think I can live without that just fine. The critical path for me, I’m pretty sure, is in the things I use and update frequently: bookmarks, Greasemonkey scripts and extensions.

Epilogue

Once I got home, I logged in and gave Dropbox time to sync up with the Mac. Once sync’d, I edited the Mac’s Firefox default profile and created the same symbolic links I’d created on my Linux machine. After restarting, the default profile loaded up beautifully here too.

I’ll keep an eye over the next few weeks, but hopefully this will be every bit as effective as my kitchen sink solution, but without the annoyances.

Subscribe19 Comments on Redux: Synchronizing Firefox Through...

  1. rob said...

    Dropbox + Firefox synchronization = hot topic these days – keep us posted on your results- at this time I’m lazy to set everything up, but maybe eventually I’ll give your solution a try

  2. Rob Wilkerson said...

    It’s working better with just bookmarks and extensions being synchronized. Sync’ing everything was just too much. There was way too much activity unless you can somehow be sure that only one instance is open at a time. Full duplex action is just too chatty and created a lot of conflicts for me.

  3. markus heinzer said...

    I will try this too. thanks for the idea! On windows: how can I create symbolic links?

  4. Rob Wilkerson said...

    A symbolic link (to a file) is, in the Windows world, a shortcut. That will work for files, but I don’t believe it will work for directories. If you have Cygwin installed you can create symlinks for any directories, but otherwise I guess you’d have to create a shortcut to each file involved.

  5. markus heinzer said...

    Thanks for the answer. I tried this all out now: it doesn’t work with shortcuts. One has to make hardlinks. On the commandline: fsutil hardlink create c:\link.file c:\original.file
    It will look like two files, but they point to the same file.

    By the way: I synchronize these files to have all passwords and form-entries and cookies synced:
    cert8.db / cookies.sqlite / formhistory.sqlite / key3.db / permissions.sqlite / places.sqlite / secmod.db / signons3.txt

  6. Rob Wilkerson said...

    That’s outstanding, Markus. Thanks for the additional information. I didn’t know about the fsutil hardlink command.

  7. GollyJer said...

    Does dropbox do delta syncing yet? I switched to powerfolder (which does delta syncing) 5-6 months ago. My entire FirefoxPortable folder is syncing with significantly reduced chatter as places.sqlite is really the only file that changes drastically between runs.

    As far as your problem with accidentally leaving firefox in one location, thus causing issues when opening in another location… Can’t someone write an addon that let’s you set a timer that closes firefox if left unattended for more than x minutes already?

  8. Rob Wilkerson said...

    @gollyjer:

    As far as I know, Dropbox doesn’t do delta syncing. I may take a look at powerfolder, though. That sounds interesting.

  9. Chris said...

    thank you for the guide. I have extensions and greasemonkey scripts syncing using dropbox. just fyi for windows vista and win 7 users.. the mklink command works wonders.

  10. Roger said...

    Once the technique is fully refined, wouldn’t it be great to package it into a Firefox extension? Is there anyone here who would know how to do that??

  11. Dave Hansen-Lange said...

    This works fairly well. But places.sqlite is still being synchronized almost constantly (about every 10 seconds, I guess everytime I visit another page). This seems like too much to me. I get the impression that it will eat up my bandwidth and dropbox space. For now I think I’ll just synch the other things.

  12. Rob Wilkerson said...

    Yeah, I notice a similar level of chatter if I have FF open on multiple machines. It doesn’t eat Dropbox space (as far as I can tell) because it’s the same file, nor should there be a bandwidth issue unless it’s a huge file, but it’s definitely annoying.

  13. tomw said...

    Take another look at XMarks (Formerly Foxmarks). Until recently I was completely resistant to the idea of syncing my bookmarks into the cloud. Until I found out that it supports the use of a “Custom Server”. So you can use your own FTP or WebDAV (via https even) server to do the synchronization with. Which is much more palatable to us security minded individuals (read: paranoid).

    Not to say that I dont want to continue syncing my forms, passwords, and extensions… I will however be moving the bookmarks out of the dropbox folder and back into the default profile.

  14. Tom Fagerland said...

    Been using this for a few weeks now on two macs and a Windows machine. Unfortunately every time I start Firefox on Windows, it deletes the 1Password extension – presumably because it’s mac only. However, I need that desperately, so I have to reinstall it every time I use a mac after using windows…
    Any suggestions?

  15. Rob Wilkerson said...

    Unfortunately, I do not. I’m also cross-platform (Mac, Win, Linux) and ended up switching to LastPass from 1Password for that reason. 1Password was great, but I needed a more available solution. Before I switched, I had the same problem you’re having, but never found a solution.

    I wish I could be more help.

  16. Dexter said...

    Hmm, well I just sync the whole Mozilla folder and use this AutoHotKey script to automatically close Firefox after 15 minutes of computer inactivity, although I only use Firefox on Windows.


    #Persistent #InstallKeybdHook #InstallMouseHook
    InactivityPeriodMins=15
    SetTimer,CheckPeriod,1000
    Return

    CheckPeriod:
    If (A_TimeIdlePhysical >= InactivityPeriodMins*60*1000)
    { SetTitleMatchMode 2 GroupAdd, Firefox, Firefox WinKill, ahk_group Firefox
    }
    Return

  17. Keriderf said...

    This is Great. Currently i’ve been syncing my whole profile on my home pc. And i have portable firefox with the profile synced up to the portable dropbox on my usb drive. But its really choppy cause my dropbox is always updating which uses a lot of resources on a usb drive and cause all the programs running off the usb drive to bog right down. With this fix it might alleviate some of my syncing stress. Something that has been an issue for me is that every time i plug into a new comp i have to update all my add-ons? Any ideas on why that is?

  18. OP said...

    My places.sql bookmarks file is currently 22MB, will this generate too much traffic to dropbox?

    I read from above that FF seems to change places.sql file every 10 seconds, requiring a complete upload of that file to server?

  19. Rob Wilkerson said...

    op –

    It can be a little verbose, but I’ve never found it to be problematic. Dropbox handles things just fine.