My research involves a combination of coding, bench work, and pencil-and-paper equation solving, and there are a couple of programs and tricks that I find myself constantly using to make my life easier when storing and manipulating data:
1. Mathematica + Dropbox. One of the more annoying features of Mathematica is its lack of a simple “undo” feature for typing errors—if you accidentally delete a cell containing an important code snippet, it is pretty much gone forever. A really cool workaround I found here involves storing all of my Mathematica notebooks in a Dropbox folder that has automatic versioning activated. I’ve inserted the snippet
SetOptions[SelectedNotebook, NotebookAutoSave -> True] into the preamble of all of my notebooks (that is, the evaluation block that I always run before I begin working on the notebook, which does stuff like set the colors of plots and control the scope of new variables). This just tells Mathematica to perform a Save every time I evaluate some portion of my code, which Dropbox in turn sees as an opportunity to create a new version of the notebook online. I found that this method was much simpler to use than the options for Mathematica and GitHub (since it really just consists of remembering to dump all of my notebooks into Dropbox), and so I use this trick almost every day.
2. NameChanger. There are a number of ways to change the names of a large batch of files using native utilities on OSX—you can create a smart folder, write a Python script, or use Automator. NameChanger is a simple, open-source Python program that allows you to change the names of hundreds of files using regular expressions and a GUI.
3. Caffeine. This is a great utility for Keynote presentations and running long SSH sessions. All that this program does is prevent OSX from going to sleep for some user-specified amount of time.
4. Illustrator+Photoshop files into Keynote+InDesign. Whenever I’m designing a talk or a poster, I make of all my figures and diagrams using Illustrator and Photoshop. It turns out that you can embed links to these files into Keynote presentations or InDesign files (I use InDesign for all my posters), and if you update the original source files, you can set the host files to automatically update the poster or talk to have the newest figures. This is very helpful for when your supervisor reminds you that you forgot scale bars on all your plots, since after modifying the images you won’t have to re-export all your figures and then re-import them into your presentation or poster.
5. IFTTT (if this then that). This is a high-level web app and smartphone app that allows me to pass data and actions between common websites, like most major social networks, my Google docs account, and my email. The design is pretty similar to Yahoo Pipes, in that it doesn’t serve a specific purpose so much as it allows people to perform a lot of actions that otherwise would require programming web scrapers. For example, I use a script (the site calls them “recipes”) that records my GPS coordinates to a Google Document every time I send an email from my smartphone. This data is part of an ongoing machine learning project I’ve been working on. There are also scripts that do a lot of common tasks, like transferring all tagged Facebook photos to a Dropbox folder, or mirroring flickr collections in a 500 px account. I haven’t yet exhausted the range of functions accessible with this site, but I can see it coming in handy for performing little tasks that would otherwise require programming a custom Python script that accesses the API of two different web applications. IFTTT has essentially made a high-level wrapper for accessing API, allowing simple data transfers and operations to be performed.
6. iPython Notebooks. These are what convinced me to switch to Python over MATLAB. The killer feature of the cell interface is that it allows you to only run or re-run portions of a script, making it competitive with MATLAB’s own cell-mode interface for scripting. But unlike MATLAB, importing new functions and packaging functions is really painless. However, I still prefer using Mathematica notebooks for symbolic math over iPython+SymPy, since the former has special syntax for typing symbolic expressions where it formats things like fractions/square roots as you type, making it easier to make sure you are typing the formula correctly.