Thread: Primitive FetchCommand

Viewers of the Kusu repository will recently have seen a flurry of activity in the sandbox. The Singapore-based team has been thinking hard about the next generation of Kusu - what we liked in Kusu now, what could be improved, and what was downright ripe for redesign.

One of the tenets of the new Kusu, was that there should be better abstraction of basic tools(which we call primitives) and higher-level executives(composites). However, that discussion is a post for another day - here I will focus on a small tool that I developed called Fetchtool.

The objective of this tool is to be a universal file/directory fetcher that, given a URI, will fetch any file, or mirror a directory, across protocols such as local file://, ftp, http/s, and even yum and yast repositories.

The CLI program, fetchtool[1], is simply a tool that consists of a ‘FetchCommand’[2] Command object(see Command Design Pattern[3]). The FetchCommand class has an attribute called PROTOCOL_HANDLER_DICT, which is a dictionary of supported protocols. So potential users can query for a list of supported protocols like this:

>>> from primitive.fetchtool.commands import FetchCommand
>>> print FetchCommand.PROTOCOL_HANDLER_DICT.keys()
['http', 'https', 'file']

To use FetchCommand is also simple:

1. FILE (fetch a file called /home/foo/bar to /tmp):

from primitive.fetchtool.commands import FetchCommand

fc = FetchCommand(uri='file://home/foo/bar',
fetchdir=False,
destdir='/tmp',
overwrite=True)
fc.execute()

2. HTTP (mirror a directory to /tmp):

from primitive.fetchtool.commands import FetchCommand

fc = FetchCommand(uri='http://nowhere/dir',
fetchdir=True,
destdir='/tmp',
overwrite=True)
fc.execute()

3. HTTPS (fetch a file to /tmp):

from primitive.fetchtool.commands import FetchCommand

fc = FetchCommand(uri='https://nowhere/nofile',
fetchdir=False,
destdir='/tmp',
overwrite=True)
fc.execute()

Fetchtool is used in other parts of Kusu 2.0 to fetch files/directories locally. It abstracts away the complexities of specific protocols like authentication and authorization to provide a unified facade that is easy to use, and also extendable through the subclassing of the ProtocolFetchHandler class[4]. That last reference also provides code to the existing file, http, https protocols.

[1] fetchtool code: FishEye: file kusu/sandbox/primitive/src/app/fetchtool/bin/fetchtool.py
[2] FetchCommand code: FishEye: file kusu/sandbox/primitive/src/app/fetchtool/lib/commands.py
[3] Command Design Pattern: Command pattern - Wikipedia, the free encyclopedia
[4] ProtocolFetchHandler class and children: FishEye: file kusu/sandbox/primitive/src/app/fetchtool/lib/fetchhandler.py