ionice: renice for your I/O operations

December 22nd, 2008

A few months ago I posted a crazy solution that would allow me to write a large file to disc without killing disk I/O and causing other production services to be impacted. I post this as a follow up to that. Shortly after making that post I learned of a utility called ionice, which takes advantage of Linux’s I/O scheduling capabilities. This utility is something akin to renice, but for disc I/O scheduling instead of CPU scheduling. I use it all the time now and the servers I manage are more stable because of it.

On the same note, I’d like to give a plug for another really nice tool called iotop. It’s a python program which gives an interface similar to top. It displays useful information about which processes are using disc I/O and how much.

Gmail unbearably slow? A minor fix.

December 19th, 2008

Over the past six month I have perceived that Gmail and Google Apps mail have become slower and slower. I’m using it on high-speed connections, so bandwidth is not the issue. I have lost quite a lot of time sitting around waiting for the Gmail interface to finish loading so that I could perform even the most basic operation. Even worse is that the way Firefox is implemented, if one window/tab has pending operations, you may not be able to do anything with other Firefox windows/tabs until those operations have completed, so I was often left with Firefox being totally wedged until Gmail would finish loading. The basic HTML interface is faster, but it’s no fun at all. Today I decided to do something about it, or at least try.

Through observation — and the interface gave me plenty of time to do it — I discovered that the overall slow loading seemed to coincide with the loading of the little star images next to each message row in a list view, which can be used to quickly flag a message as important. The image itself is transparent image called cleardot.gif and is used in a number of places. It is just a placeholder that the interface uses to mark where an icon will actually go, and then which icon gets displayed in that location is handled with CSS via a background image.

In any case, I noticed that if I disabled images altogether for the domain mail.google.com, that the interface speeded up absurdly, and was still quite usable. However, there were a few images that I didn’t want to go away. What I ended up doing was adding the offending image URL to my Adblock Plus rules. The web interface is now at least 50% faster for me, if not more.

The main things I notice as missing now are the paper clip icon indicating an attachment in the list of messages, and the star icons next to each message row. Neither one of those things was really necessary for me. I can still flag messages as important, just not through that icon, and attachments still show up in messages, I just can’t know if a message has an attachment until I open it, which is an acceptable loss for me for the incredible speedup of the interface.

The offending image URL for regular Gmail is: https://mail.google.com/mail/images/cleardot.gif. No big deal, it’s small, but sometimes it had to fetch that same image dozens of times to complete the interface.

Grpahical GPRS utility for Openmoko SHR distro

December 15th, 2008

This past weekend I flashed my Openmoko Neo Freerunner to the software distribution known as SHR. Immediately I was very pleased with the look, and after playing with it most of the weekend, I’m now attempting to use it as my primary phone.

GPRS was working, but I wasn’t satisfied with any of the current graphical tools to launch it, of which I only knew of one, and launching at the command line is unacceptable, as things on a handheld device need to be finger-friendly.

With that in mind I wrote a small graphical utility to launch GPRS on FSO based distributions. The tool makes use of Gtkdialog, which wasn’t in the SHR repositories, so I had to build the ipk as well: gtkdialog-0.7.9_0.1_armv4t.ipk.

At one point I had 4 different applications accessing the Internet at once: a browser surfing the web, an active chat via Pidgin, a terminal with an SSH session to a remote host, and tangoGPS downloading map tiles. Granted, none of it was super fast; GPRS is old, slow technology, but it’s still usable for many things. And over all that I was still able to receive a phone call. Things are getting better by the day for Openmoko Neo owners, but there is still a way to go.

Less is More

November 8th, 2008

A couple days ago Mike Linksvayer left a comment on a photo at my Flickr photostream. His comment includes a link which references a blog post by Mark Pilgrim. In this blog post “The pursuit of happiness,” Mark gives an 8 step process to achieve this goal. It essentially consists of getting rid of all your worldly possessions. By his scale, I’m more or less at step 6 and I don’t have plans of going farther than that, and I consider myself a somewhat extreme minimalist. He claims he’s still on step 4, so I’d like to chat with him once he reaches step 6.

Anyway, as Mike divined, this is a matter very close to my heart. I am absolutely convinced that there is an inverse relationship between the amount of things one owns and the probability of happiness, though I would not, myself, use the word “happiness,” rather I would say joy or contentedness.

Every thing you own is attended by some amount of complication. I hear a lot of people continually talking about how busy they are, how tired they are, how complicated their lives are. Generally it is believed that all this complication and weariness will be solved by some acquisition or other or [useless] bit of technology or money. Sadly, none of this is true and so people have a propensity to continue piling complication on top of complication in the vain hope that they can fight fire with fire. Incidentally, I’m not even sure that most people realize they are fighting fire with fire.

There is a common aphorism in English that goes something like this: it’s better to have it and not need it, than to need it and not have it. This concept is more insidious than most people imagine — a recipe for complication, for you can never have everything you are going to need and you will rarely, if ever, need everything you have.

A number of years ago I had some free time and some spare money and was on the verge of hiking the Pacific Crest Trail. To that end, I bought a book called “The Pacific Crest Trail Hiker’s Handbook” by a man named Ray Jardine. In this seminal book he comments on the aforementioned principle and turns it on it’s head by producing his own: if I need it and don’t have it, then I don’t need it. Not only did this notion change my idea of backpacking, but it crept into my daily life. From it, I generated a new principle: if I have it and don’t need it, then I get rid of it. Since that time, sometime back in the late 1990s, I have whittled down my belongings to what can be carried on my person. Sure, at any given moment I may have more than that, but nothing that I won’t readily leave on the sidewalk for the taking or at the Goodwill.

In the avoidance of gaining new things, I also abandoned gift giving on holidays and birthdays. For the vast majority of people this type of gift-giving is a keep-up-with-the-Joneses, perfunctory gesture of the most unsentimental variety. When eight or nine years ago I announced that I wanted not one thing for my birthday or any holiday and that nobody should expect any from me, well, this announcement was met with some amount of disbelief and chagrin. I was told: but you are throwing out the baby with the bathwater. To which I replied: but you are drowning the baby in the bathwater.

So, for the better part of decade now I have happily avoided commercialized gift giving/receiving. Not only have I managed to avoid accumulating a lot of things I didn’t want in the first place, but my life is much less complicated as a result of it all. Gift giving is exceedingly special, just take care to do it as a result of your heart, and not the calendar. I don’t mean to imply that these two things are necessarily mutually exclusive, only that one needs to be cautious when the two mix.

Summary: less is more. Do you want to uncomplicate your life? Start divesting yourself of material possessions, stem consumerism, live as consciously as possible. Live free.

Now, what I want is, Facts.

November 3rd, 2008

Here is a wonderful excerpt from a lesser known book by Charles Dickens called “Hard Times“. Every now and then I go looking for this bit, and so I post it here for my own benefit, as well as by way of a recommendation of this book. It is rather a scathing indictment of many of the driving principles of Victorian England. Dickens executes this polemic with a wry acerbity that makes me laugh out loud:

Thomas Gradgrind, Sir. A man of realities. A man of facts and calculations. A man who proceeds upon the principle that two and two are four, and nothing over, and who is not to be talked into allowing for anything over. Thomas Gradgrind, Sir—peremptorily Thomas—Thomas Gradgrind. With a rule and a pair of scales, and the multiplication table always in his pocket, Sir, ready to weigh and measure any parcel of human nature, and tell you exactly what it comes to. It is a mere question of figures, a case of simple arithmetic. You might hope to get some other nonsensical belief into the head of George Gradgrind, or Augustus Gradgrind, or John Gradgrind, or Joseph Gradgrind (all supposititious, non-existent persons), but into the head of Thomas Gradgrind—no, Sir!

I nicked the excerpt from a page at randomhouse.com. And there is more where that came from.

Breakfast smoothie

October 26th, 2008

I haven’t posted in a while because I’ve been in a state of transition — moving from San Francisco to South Florida. I’ll write more about that soon, but for now I must write a few words about one of the most important things in my life: smoothies. I haven’t had a proper smoothie in weeks, and having just moved into my new apartment I was able to make one this morning. I was so impressed that I was inspired to write about it.

For quite a number of years now my breakfast has consisted of a smoothie. I’ve had plenty of time to refine the recipe and I share it here. It’s nothing complicated, but it’s pure goodness in a glass:

  • 1 medium banana
  • 1 small apple (or 1/2 of a large apple), preferably fuji or braeburn
  • about 1/2 cup of mixed berries
  • 1 tbsp of flax oil
  • 1 tbsp of green vegetable powder
  • 1 or 2 tbsp of peanut butter
  • 2+ cups of water (or enough to make the consistency you want)

I don’t mention them above, but there are brands that I really have come to appreciate:

  • Flax oil: Spectrum flax oil with DHA
  • Peanut butter: Santa Cruz lightly roasted creamy. All peanut butter is not created equal. I once tried Whole Foods own 365 brand organic peanut butter and it wasn’t half as good as that from Santa Cruz. Don’t even think about Jiff, Skippy or any other of those vile peanut butter wanna-bes.
  • Green vegetable powder: Amazing Grass GreenSuperFood. There’s nothing specific that I’m looking to get from this ingredient, but my reasoning is that a freeze dried powder of all sorts of green vegetable matter can’t but be packed with loads of valuable nutrients.
  • Mixed berries: Whole Foods 365 branch frozen berry Medley. Fresh berries would be better, but this is acceptable to me and it adds a little chill and thickness to the smoothie.

I would normally never have toast for breakfast, or at any other time, no matter what the USDA may recommend. In my opinion the food pyramid is totally flawed at best, a health hazard at worst. However, a friend of mine, Cabe, introduced me to some bread that is make with only sprouted grains, no flour. The brand is Food For Life “Ezekial” bread. It can be found at any Whole Foods, and I even saw it at a Publix recently. I toast two pieces of the bread and top it with some vegan, organic vegetable spread.

I eat this meal every morning for breakfast and I never tire of it. I eat it around 7AM to 8AM and I don’t find myself hungry again until around 1PM. The vegan salad I eat frequently for lunch is another post.

Update (Thu Oct 30 09:58:56 EDT 2008):
My good man Jon Phillips was kind enough to repost on his blog the recipe for this smoothie.  However, someone left a comment about having tried it and said that it tasted like “cat sick.”  However, this person didn’t follow the recipe, and I realized that I should make a few other observations about the smoothie to spare others from drinking “cat sick” for breakfast:

  • Use a electric blender.  You cannot blend an apple and frozen berries by hand.
  • Do not substitute the peanut butter with Vegemite.
  • Do not use cranberries, or use very few of them.  Cranberries are great, but they are bitter.  Use berries that are sweet.  My mix contains strawberries, blueberries, blackberries and raspberries.
  • The green vegetable powder has a tendency to impart an “earthy” flavor to the smoothie.  I have found that this is largely tempered by what type of berries are used and in what proportions.  In my experience, the more strawberries, the sweeter the flavor.  If the vegetable powder is too strong a flavor, use less of it or omit it altogether.

dd saturating disk I/O

August 15th, 2008

Today I was in need of pre-allocating a 50G file on a number of CC‘s servers. I started off with this:

# dd if=/dev/zero of=bigfile.bin bs=1M count=50000

However, that operation was too expensive in terms of disk I/O and was causing critical web services to fail. I played around with multiple options in dd but to no avail. I finally ended up doing this, which works, but seems rather ridiculous. Surely there’s a better way to do it:

# for i in $(seq 1 50000); do nice -n 19 dd if=/dev/zero of=bigfile.bin conv=notrunc oflag=append bs=1M count=1 &> /dev/null && sleep 0.1; done

The nice probably isn’t necessary, but I threw it in for good measure.

Openmoko Neo Freerunner: Round one

August 10th, 2008

I received my Openmoko Neo Freerunner last Monday, August 4, much earlier than I had anticipated. Score one for both the Canadian postal system and the USPS. I haven’t had one big chunk of time to experiment with it, but a few hours here and there over the last 6 days. I’ll start with some general impressions first.

This device is fun. Any phone that I can SSH into is a phone that I’m bound to like, even if it doesn’t work perfectly. Don’t compare this device to an Apple iPhone. It does a disservice to Openmoko as well anyone who might mistakenly get the impression that this is some sort of iPhone alternative; it is not.

I won’t even bother talking about my experiences with the device prior to the official August 8th release of the Om2008.8 distribution/image. Before installing this official release the device was only semi-functional in any given area. That said, I have to note that I was still loving it.

So, the Om2008.8 release is actually pretty good. All aspects of the phone that would matter to me seem to work nicely, SMS messaging works well, the device suspends after some inactivity and then wakes upon receiving a call. The interface is fairly nice, intuitive, and even reasonably snappy.

One thing that surprised me is that GPS functionality wasn’t working out-of-the-box. I had to manually install gpsd via opkg and also install tangoGPS.

The device is mostly functional but still quite buggy. At the present moment this phone will probably not serve as a primary phone for most people. However, this will probably all change over the next number of months. If you are interested in being an early adopter, buy this phone now. If you want a smartphone that 100% works, wait a while. I’m very happy with my purchase, and look forward to seeing how the software progresses over the next, say, 6 months.

Anyone interested in this device owes it to themself to subscribe to community@lists.openmoko.org and support@lists.openmoko.org. Those lists are somewhat busy, but have proved invaluable.

Soon, I’ll probably create a page that describes various configurations that I’ve done, fixes, and general notes on getting things working.

foodary.org

August 2nd, 2008

Other than having posted about nutridb.org a couple weeks ago, I haven’t really thought too much about the site or code in a while. However, once every couple months it occurs to me to search Google for “nutridb” to see if anyone else has mentioned it. Today I did one such search, and was surprised to find a site that was using the nutridb code: foodary.org. There are a couple of other sites associated with this one: foodary.wordpress.org and foodary.com.

I was pleased to see that the author, Keith Taylor, was nice enough to credit me at all three sites. This is the type of usage I was hoping to see … someone taking the code, putting a prettier shell around it and perhaps extending and improving it.

At the moment he appears to have just modified the CSS and added a header image, but this is fine. Viva free software. I released the PHP code under an MIT (X11) license, and I generally prefer this license type, along with BSD licenses, to the [L]GPL because I consider them to truly be free in every sense of the word, though I do think there are certainly cases where the GPL and LGPL are warranted.

Openmoko Neo Freerunner on the way

August 2nd, 2008

A couple days ago I took the plunge and ordered an Openmoko Neo Freerunner. It is generally considered to be a smartphone, but I’m viewing it more as a pocket-sized portable computer than a phone. The Neo Freerunner is nowhere near as functional or elegant as an Apple iPhone, for example, but it’ll be a cold day in hell before I ever own an Apple product.

The Freerunner is built upon a free and open software stack, including a Linux kernel. Even the hardware designs are open. Interesting for me, being a CC employee, is that the CAD files for the case are released under a Creative Commons BY-SA license.

At $400, plus $40 shipping from Canada, the device was a bit pricey, but for an unlocked touch screen phone with a lot nice features, and that runs Linux, I was happy to pay that amount, if for no other reason than to support such an organization.

I’m sure I’ll post more here about my experiences with the device once it arrives (hopefully by 2008-08-11).