View Original Text

Hide Table of Contents

Lisias' Confederation of Micro Services for Retrocomputing

Because Napoleon may had failed to conquer the World - but I will NOT. =P


net.lisias.retro (Introduction)

The net.lisias.retro Micro Services (I still don't know how to name the thing!) are a (growing) set of Micro Services (and a Standard for integration of such services) to provide services to the Retrocomputing community.

Aiming to be supported from modest, embedded appliances to full blown servers, the Standard is language and framework agnostic: it can (and should) be implemented on any capable platform. See the Documentation for more technical information.

Currently it's being hosted by three Raspberries Pi (a 256 V1.0 and a 512 Mb V1.1, both Model B, and an additional 2014 Model B+) from a stand on my living room, and the following services are currently provided:

net.lisias.retro.file

A Distributed Search Engine (and more) Solution for retro computing files.

Divided into two main components, the Service and the HTML Client.

net.lisias.retro.file.search.WS

A File Repository Search Engine for Retro Computing made in Python.

Currently served by httpd2.bash and hosted by a Raspberry π 2011.12. It was used to be over the top of my (messy) table, but I finally get it together and built a nice place on that dead space on a stand I have in my living room.

Operating Servers:

Full article (pt_BR).

net.lisias.retro.file.search.HTML

The search.WS Web Service aims to be minimal, being able to run on limited computing resources. Intended to run on Classic Home Computers (when possible), on Raspberries π (2011.12 was used for development) and similar devices.

That said, the service just don't handle Presentations. Point.

So I had the need to provide a client so people can use the thing. I ended up being forced to deal with that CORS thing, what didn't made me happy.

These artifacts are the answer for this need.

Operating Server (Raspberry π)

Full article (pt_BR).

net.lisias.retro.file.mpd.ws

Murphy was a Prophet, I acknowledge that. Another unwelcome holiday plays havoc with my productivity and I end up improving the Shoutcast service I started on the Rπ. Now this Web Radio can be remotely controlled by Web Service. :-)

Full article (PT_BR). (pt_BR)

net.lisias.retro.db

net.lisias.retro.db.query.WS & HTML

Besides searching for files, a Retrocomputing solution also need to provided organized, not rarely relational, data about such artifacts.

One solution for this is the ZXDB database, from Einar Saukas.

As with the Search Engines, instead of instantiating a huge server aimed to withhold the worst possible load, and then pay for it 24/7 (including when nobody is using it), a distributed approach is being developed - this is Research in Progress, please take all of this with a huge grain of salt. :-)

Again, a myriad of volunteered appliances would serve the queries - and as the load increases, more appliances would be confederated in order to keep serving the intended public.

Originally implemented for MariaDB 10, I converted it to Postgresql 9 using a in-house "transpiler" and put both online. I intend to compare both solutions head to head using exactly the same hardware and the same dataset.

Again, all of this is far from being even Alpha. :-)

Operating Server (Raspberry π)

Full article (pt_BR).

net.lisias.retro.proxy

As nothing is so bad it can't get worse: a tropical storm, typical this time of year here in São Paulo, dropped me without Internet all day. As I could not do anything serious, I took a part of the day to pursue one of feedbacks I got for search.WS.

A Retrocomputing Scene colleague Popolon Campus suggested a cooperative network of Rπs (or any similar appliance) to serve the searches.

The suggestion was more than excellent, as the preferred targets for the search engine are limited computers and/or dirty cheap appliances that plague the Scene Meetings (which do not always have internet available !!), making clear the fact that just one appliance cannot serve all datasources. The first incarnation of search.WS exhausted an Rπ 2011.12 in just 5 days of development!!!

The suggestion evolved fast to what I'm calling a "Confederation of Providers". A proxy.WS (this) concentrates the requests from customers and, through internal tables, redirects the requests in a round-robin regime to Providers that register themselves as supporting some (or all) the requests.

To support this solution, Providers need to provide metadata about the services they provide (ugh... terrible phrase, uh?).

net.lisias.retro.proxy.WS

Operating Server (Raspberry π)

Full article (PT_BR). (pt_BR)

Confederation

A (now) sophisticated mechanism is implemented to allow a Provider to sign up into a Proxy in order to provide their services. A Service can be pulverized into many Providers (i.e., each Provider can serve just a few Entities or Verbs from a Datasource, with others filling the gap - and the Proxy does the magic of making everything work tight).

The Proxy, so, keep updated a list of Providers and its capabilities, and then receive client requests, redirects them (or makes himself the request echoing back the result) to a capable registered Provider that so serve it.

Since each Provider may be part of more than one "Confederative Unit" (whose pivot is the Proxy), we have somewhat a semblance of a Confederate Union - allowing Providers to enter and leave the pool without customers needing to adapt themselves.

Singleton services were implemented. A Singleton Service is a service those instance must be unique on the Confederative Unit (perhaps in the whole Confederation?). A concrete example is the WebRadio described below.

Full article (pt_BR).

Pics, or it didn't happened! :-)

One of the earliest screenshots from the Search Engine Front End:

Screenshot

And here the first setup of the "server pool" - good ol'time when everything fitted in just one 512M Raspberry π (yes, the hardware below the Rπ0 is a DE-2 FPGA board, and the rightmost monitor is a TV set!):

First setup

This was the inception of my local "datacenter", one 256M Rπ and a 512M one being held together by an OpenWRT router:

Old setup

This is the current setup: one 256M Rπ, and two 512M:

Current setup

This last setup is already overloaded. I realized that it's not cost effective (here at Brazil) to stack up a bunch of Rπ's . The older, cheap ones are not available anymore and the new ones don't have all the memory I need for some datasources - not to mention the horse power to carry on some tasks.

So, and, embracing the Affirmative Action for the defense of the Deprecated, this is 4th Raspberry π being prepared to enlist the Confederation:

The 4th Rπ

Check your privilege =D . It's perfectly reasonable for this machine to be considered a Raspberry π if it feels like one and costs me the same! X-P

(and yes, this little monster cost me LESS than a new bare bones Raspberry π 3!!!)

PROJECT TIMELINE

Or... What a man does to avoid house keeping on extended holidays!

The full history of the project can be found here.