Tonight I wrote the disk caching logic (well most of it). zmugfs will now look in the memory cache for the imgdata, if it doesn’t exist it looks in $HOME/.zmugfs/cache for the file. If it can’t find it there, it does the final attempt to read it from smugmug itself.
I haven’t figured out how I’m going to calculate the size of the disk cache, that’s the last step to finishing that feature, otherwise it’ll use whatever disk space you have.
That’s it for hacking tonight. Off to watch TV.
Yesterday I ran into a slight road block with simplejson. But with some playing around in the python interpreter I was able to see what the problem was with how simplejson 1.7 calls object_hook.
Take for instance the following string:
‘{”foobar”:”baz”,”comp”:{”key”:10},”array”:[10,20,30]}’
simplejson 1.3 would pass the above string into the object_hook as is. With 1.7 {u’key’: 10} is passed in first, followed by {u’foobar’: u’baz’, u’array’: [10, 20, 30], u’comp’: None}
Unfortunately, when {u’key’: 10} is passed in we have no context as to what this is.
I know, you all want to know what my solution was to the problem. 
Well, it was quite simple, I continue to look for the key “Categories”, but if the argument does not contain it, I return the string unaltered. That’s it. Now zmugfs works on Fedora Core 6 and Fedora 7.
I tested out zmugfs on Fedora 7 today and it completely blew up. The objecthook function being passed into simplejson.loads() behaves differently. On Fedora Core 6 (using python 2.4.4 and simplejson 1.3) my objecthook method is called once with the entire JSON response which I can parse and return a Tree object. On Fedora 7 (using python 2.5 and simplejson 1.7) it gets called 152 times with each call being a subset of the original response.
I really would like it to run on Fedora Core 6, but it’s pretty dead since Fedora 7 has been out and Fedora 8 will be out soon enough.
So will zeus abandon FC6 support for something new like F7 or F8? Or will zmugfs be updated to work on all Fedora’s? Stay tuned for more zmugfs on zeusville. Same zmug channel, same zmug time!
<insert cheezy soap opera music here>
Tonight I finished a really lame in memory cache to store the image data. Basically it’s a dictionary keyed by the path (filename) and the values are another dictionary with image data and timestamp. There is a new configuration entry “image.memory.cache” where you list the number of images to cache in memory, defaults to 5 (about 10MB). When it hits the max size, it removes the oldest entry in the map.
Next up is disk caching. I plan to cache some of the images on disk (in $HOME/.zmugfs/cache) I could probably also use /var/cache/zmugfs for a global install but for the first cut I’ll stick with the former. This adds yet another configuration entry “image.disk.cache” specified in megabytes. Yes, I know it’s lame that the value for the two caches are specified in different units. But the idea of dealing with file sizes in the memory caches versus number of entries was not appealing.
After the disk caching, the only thing left is packaging and some setup scripts to make it easier to create your configuration file.
We’re getting very close to releasing zmugfs 0.1, the read-only release. As you may recall on Monday I got zmugfs to show imgdata in nautilus.
But I could only get it to work if I used the thumbnail sized imgdata. In the read() method I was logging into smugmug, reading the thumbnail imgdata, logging out of smugmug, getting the imgdata via http GET, and returning. Well, this is crazy because read gets called multiple times with different offsets. So I ended up implementing the open() and release() methods. In the open, I get the imgdata (original size) via the smugmug api and store the imgdata in an image cache (an in memory dictionary). In the read method, it now respects the offset and size attributes by reading from the image cache. Then in the release method, I delete the entry from the image cache. I know not much of a cache. But it solved my problem. I’m putting the cache work off a bit, not sure how to approach that yet.
Next thing I wanted to try all of the command line utils (at least the ones I’ve used before) to see how they react to zmugfs. I was pleasantly surprised with the result.
| command | result | comment |
|---|---|---|
| basename | SUCCESS | |
| cat | SUCCESS | |
| chgrp | FAILURE | chgrp method not implemented |
| chmod | FAILURE | chmod method not implemented |
| chown | FAILURE | chown method not implemented |
| cp from | SUCCESS | able to copy image from smugmug using zmugfs |
| cp to | FAILURE | mknod not implemented |
| file | SUCCESS | got appropriate file type |
| ln -s | SUCCESS | softlink created |
| ls | SUCCESS | listing appears correctly |
| mkdir | FAILURE | mkdir method not implemented |
| mv | FAILURE | copy part worked; unlink not implemented |
| rm | FAILURE | unlink not implemented |
| rmdir | FAILURE | rmdir not implemented |
| touch | FAILURE | setattr not implemented |
From a readonly perspective, it is looking great. What’s left before a release you ask? Just a few things.
That’s it, once I finish those things zmugfs 0.1 will be released out into the wild! I’m really excited as this is my first ever open source project. I’ve submitted patches to a few things, but never anything this extensive.
Tonights zmugfs update is permissions and time are now shown correctly. I defaulted the user and group to that of the user running the program (in this case me) And I now show the date and time correctly from the api.
Update: by “in this case me” it doesn’t mean I hardcoded my uid and gid. I used os.getuid() and os.getgid()
By the way, disregard the drew and foo entries. Those are debug strings which will be removed.
Success. I got zmugfs to show image data in nautilus. It is a horribly inefficient algorithm but it works. I do the following:
I know EWW! But it worked!
That’s cool and all, but I bet what would be really cool is to open the file up in GIMP.
How’s that for coolness! It’s 12:18am EDT, I need to get to bed because I have to get up at 6:30am to get the kiddies ready for school. If all goes well, I should be able to get zmugfs 0.1 released by next week.
Yesterday I started implementing the image listings for the albums. I finished it up today. Here are the latest screenshots.
First up is an ls listing of the Album which appears on the filesystem as a directory (click the image to see a larger view, and yes that is Obi-wan in the background). The permissions aren’t quite right for the images, but that’s a minor change to the stat structure.
Next up is a subcategory containing albums. If you recall before I didn’t show the image counts, now the image counts appear.
Finally, you can see an album with the list of images and their file sizes. Since this only returns the filename information nautilus can’t render the previews. Still this is cool stuff, well in my opinion it’s cool stuff
Almost a month ago I posted a screenshot of zmugfs via nautilus. Back then I was returning a hardcoded list of strings with no inode information. Last week, I got the read-only listing of categories and subcategories to work, see the ls output here.
I thought to myself, I bet folks would love to see the new read-only functionality through nautilus. Well, prepare to be amazed and bedazzled. First up is a screenshot of a category showing all of the subcategories contained within. Notice all but one says “0 items”, that’s because I haven’t figured out how to get the image count for the albums contained within yet.
The Birthday subcategory contains 9 items (all albums). So if you double click on that folder you are presented with a window showing all 9 albums in the subcategory.
Isn’t that cool? You may be asking, “how do I know he’s not just returning some random list of strings?” Well, let’s look at smugmug to see what’s actually there:
Next step is to get Albums to show their images and their image counts. Once that is done, I will release zmugfs 0.1 out in the wild. The following releases will add write support and possible performance enhancements.
Func is an interesting project that solves some important problems.
It was recently released after only two weeks of development. That to me is amazing, how a few people can get together and accomplish so much. Kudos!
I wanted to try out a simple feature: get the status of my httpd process, and bounce it. Here’s a summary of what I did:
For more information on Func setup see the wiki:
https://hosted.fedoraproject.org/projects/func/wiki/InstallAndSetupGuide
Here’s a screenshot of my overlord (top window) run /sbin/service status httpd and /sbin/service restart httpd on the minion (bottom two windows). The bottom most window shows the funcd log in DEBUG mode and the middle window is the httpd error_log.
That was great. Now I can setup my kickstart files to install func on all my machines so I can control them with func. The Func wiki covers how to provision with Cobbler.
