More Filesystems

In my recent post I’ve mentioned that the Filesystem library can work on different kinds of filesystems. In this post I am going to walk you through the supported filesystems one by one.

Before you proceed with this hands-on blog post, please update to a patched and unofficial version of the Filesystem package. In the meantime some bugs and minor issues got fixed that are not integrated into the official release yet.

 Gofer new
renggli: 'fs';
package: 'Filesystem';
load.

Disk Filesystem

The Disk Filesystem is implemented in FSDiskFilesystem and its platform specific subclasses. As we have seen in the last post the singleton filesystem instance for the current platform can be retrieved using:

 disk := FSDiskFilesystem current.

Subclasses of FSFilesystem implement many methods, but we should resist from calling most of them directly. These methods implement the low-level behavior of the respective file-systems and are private to the framework.

As we have learned in the previous post we should only work with references. Filesystem instances know two methods that return an FSReference object. This is true not only for the disk filesystem, but also for all other filesystem types presented later on:

 disk root.                               " a reference to the root directory "
disk working. " a reference to the working directory "

Given a reference we can navigate to another reference in the filesystem with the method #resolve:. Resolving works similar the command cd (change directory) on Unix and Windows and returns a new reference to a file or directory. Print the result of evaluating the following expressions:

 disk working resolve: '/'.               " the same as 'disk root' "
disk working resolve: '.'. " the same as 'disk working' "
disk working resolve: '/home/renggli'. " an absolute path to a directory or file "
disk working resolve: '../bar'. " a relative path from the working directory "

Note that the message #resolve: is also understood by FSFilesystem itself. Do not call this method though, it is private and does not return an FSReference as you might expect.

Memory Filesystem

The memory filesystem is another simple filesystem type. Think of it as a virtual in-memory filesystem, very much like a RAM disk that lives in your Smalltalk image. The use of a memory filesystem can be very convenient for testing your filesystem code, because it does not pollute your hard disk and is garbage collected as soon as you do not reference it any longer. To instantiate a memory filesystem evaluate:

 memory := FSMemoryFilesystem new.

On this filesystem you can do everything you learned before. A memory filesystem is initially empty:

 memory root children size.               " --> 0 "

To create a file we can use the same techniques we learned previously:

 (memory root / 'foo.txt')
writeStreamDo: [ :stream | stream nextPutAll: 'Hey Memory' ].
(memory root / 'foo.txt') exists. " --> true "

We can also copy files from a different filesystem to our memory filesystem:

 cache := disk working / 'package-cache'.
cache copyAllTo: memory root.

The above code copies all the files in the package cache of your Pharo installation to the memory filesystem. Before you try it out make sure that you don’t have your MP3 collection in that directory, otherwise your image might blow up.

In my case I have now 64 files in the memory filesystem:

 memory root children size.               " --> 64 "

As you would expect we can perform other operations on our virtual filesystem, for example delete all the files that start with the letter F:

 memory root children do: [ :reference |
reference basename first = $F
ifTrue: [ reference delete ] ].

As you see, there is nothing special about a memory filesystem. It behaves and understands exactly the same messages as the disk filesystem does.

ZIP Filesystem

The ZIP filesystem represents a ZIP Archive that resides on another filesystem. To create a new archive instantiate the ZIP filesystem with a reference of a ZIP archive:

 zip := FSZipFilesystem atReference: disk working / 'cache.zip'.

Contrary to other filesystems a ZIP filesystem needs to be opened (and closed) explicitly:

 zip open.

Apart from that, the ZIP filesystem behaves exactly the same way as the other filesystems we learned up to now. To copy the contents of the memory filesystem to the ZIP archive we can evaluate the following code:

 memory root copyAllTo: zip root.

To enumerate the contents we use:

 zip root children
do: [ :reference | Transcript show: reference basename; cr ].

To flush the ZIP archive to the underlying filesystem we simply close it:

 zip close.

This is a convenient way to access archives. Again your code does not have to worry about the details of this particular filesystem, but transparently accesses and modifies it using references.

cURL Filesystem

The cURL filesystem is an experimental extension to the Fileystem framework. It uses the cURL plugin written by Danil Osipchuk to work with filesystems that can be accessed through FTP, FTPS, HTTP, HTTPS, SCP, SFTP, and TFTP.

First, we need to load the extension packages:

 Gofer new
renggli: 'fs';
package: 'Curl';
package: 'FS-Curl';
load.

Note that the cURL filesystem also requires the latest version of the CurlPlugin. Make sure that your VM is up-to-date before you proceed:

 Curl curlVersion.                        " --> 'libcurl/7.19.4 OpenSSL/0.9.8l zlib/1.2.3' "

What about downloading the latest cURL plugin for the Mac VM from within Pharo? To do this we can connect to the directory with the latest experimental code of John McIntosh:

 ftp := FSCurlFilesystem url: 'ftp://ftp.smalltalkconsulting.com/experimental'.

With the resulting filesystem you can do all things you already know. If you are not authenticated however, it is unlikely that you are allowed to write (or upload) to the server. Note that currently enumerating the contents of a directory only works for FTP and SFTP servers. Due to limitations of the CurlPlugin it is furthermore not possible to create directories, delete or rename files. Hopefully that will be fixed sometime soon in the plugin code.

 ftp working children.

To download the curl plugin and save it to your hard disk you can use:

 ftp working / 'CurlPlugin.1.1.0.bundle.zip' copyTo: disk working / 'CurlPlugin.1.1.0.bundle.zip'.

Unpacking the zip archive should be a breeze.

Posted by Lukas Renggli at 9 March 2010, 10:08 pm with tags filesystem, pharo, smalltalk, tutorial link

Comments

This is awesome! Thanks a lot Colin and Lukas! Looking forward to see that framework ported to many dialects!

Posted by Markus Gälli at 10 March 2010, 9:51 am link

This looks fantastic to use, but when doing a ’Curl curlVersion’, I receive a #primitiveFailedinvalidCurlHandle error :( I’ve put the dll files in the vm directory, but am guessing I need to use a different version of curl?! Currently it is 7.20.0. Should an earlier version match up better with the curl plugin?

Posted by Brian at 10 March 2010, 11:53 pm link

Looks like your version of the plugin is outdated. Sorry, I have no idea if the Windows VM has an updated version of the plugin and where you can get it.

Posted by Lukas Renggli at 16 March 2010, 5:16 pm link

Thanks for the informative posts about Filesystem.

I played with it on the Mac using the latest Squeak 4.2.3beta1U VM and your omnibrowser.image. I had a problem with encoding of file names with German umlauts. FileDirectory handles them correctly, though.

I guess this is not the ideal place for a bug report. Do you happen to know where Colin likes his? ;-)

Posted by Bernhard Pieber at 19 March 2010, 8:40 pm link

Yes, I can confirm that problem. I created a test case and submitted it to my repository. Colin regularly pulls my changes. You can do so too and ask him to pull your tests/improvements from your repository.

Posted by Lukas Renggli at 21 March 2010, 12:31 pm link