Hide Table of Contents
"Time is the school in which we learn, Time is the fire in which we burn." -- Delmore Schwartz
- 2018-02-06 late: RELEASE 1.1 HELLRAISER (re-reissue)
- Dumb mistake detected (and fixed) that revealed yet another minor, but important, feature uncompleted.
- I really need to pay more attention to my own test plans. =/
- 2018-02-06 early:
- Some tests using Raspian Stretch gave positive results.
- I still can get rid of SystemD without rendering the machine unusable for the Confederation
- But Python 3.6 isn't available yet, and frankly it's the Python 3.6 the only reason it would worth the pain the migration, so...
- Forking, refactoring and adapting FAT32lib to be used by the File Miners.
- Using droiD64 to read Commodore 64 disk and tape images.
- 2018-02-04: RELEASE 1.1 HELLRAISER (reissue)
- Some small bugs I left unfixed
- I need to pay more attention to my own test plans. =/
- 2018-01-31: RELEASE 1.1 HELLRAISER
- Project formally promoted to Beta. #HURRAY!
- SDK was updated for the new Metadata format.
gevent version to 1.2.2 on
setup.py, since the guys published a beta version as current that is broke for Python3 =/
- 2018-01-25 a 30:
- Since I royally screwed up the deploy for what would be the new RELEASE 1.1 and marvelously botched all my scheduling for the period, I proceeded with the heavy refactoring tasks - advancing tasks that were planned for 1.1.1 or even beyond :
- Annunciators - the guys that manage who is registered with (or on) who.
- Shutdown - assuring Context persistence at controlled system shutdowns (as reboots).
- One more step ahead to the goal of a Confederated Proxies pool (really, a fully functional Service Bus!)
- Heavy adjustments on Serialization:
- In special, on Enums and "hardcoded" data.
- Matadata standardization.
- 2018-01-24: RELEASE 1.0.8b
- Catastrophic failure while deploying what would be the Release 1.1 . ROLLBACK!
- 1.0.8 redeploy, with added support for the new infra (based on RPi3).
- Small adjustments to satisfy the changing environment.
- 2018-01-02 a 20:
- Pornographically tremendous overhaul on many subsystems:
- Engine framework.
- Web Services framework.
- The 'Protocol' was promoted to beta!
- The Metadata were, at least, normalized and standardized.
- Interface breaks are not expected anymore (but it still can happen).
- 2017-12-01 to 20:
- Search Engines (Front & Back End):
- Disk image contents is now being indexed! #HURRAY!
- A "Cabinet" File (as I call the file that contain files inside) is handled as a directory and its contents is indexed as regular files.
- However, the link for the download is directed to the owner URL. You search for what you want, but needs to download the whole shebang.
- Some internal idiosyncrasies were fixed (that file extension versus file type issue).
- HTML Clients were fixed.
- SDK examples didn't needed updating.
- Currently only available for Apple II Disk Images, and so only Asimov is taking advantage from this.
- Memory consumption is going to the roof, however. =/
- And yes, it took me a long time to implement this... Properly! :-D
- One more Search Engine: Vireola MSX!
- WoS Repository remains unaccessible, and the only known official mirror was deactivated.
- The Wos Archive, for now, is returning dead links.
- There's no dead line, until the moment, for a proper fix.
- Archive.org will probably save our sorry arses...
- But I have no time right now to solve this for the next release.
- Data Miners
- Better handling for repositories synchronized by "brute force"
- It was realized that some "dead" repositories are, in fact, still alive and, so, they're now synchronized regularly.
- Adjustments and enhancements on nvg.cpc .
- Implementing support for the new Search Engine functionalities.
- Activating (almost) all ready to go Archives but that were dormants by lack of infra, #HURRAY!
- Tremendous overhaul on many subsystems:
- FTP support.
- Producers, Consumers & Synced Consumers for datasets.
- Synced Consumers:
- Check ftp.service.retro.lisias.net lokking for updated
datasets and download them automatically.
- Really better visualization!
- Preemptive concurrency.
- Better tasks management.
- Individualized (and customizable) Context for each task, including controlled access (by mutex) to STDON/STDOUT/STDERR (extensible to other resources if desired).
- Contexto individualizado (e customizável) para cada tarefa, incluindo acesso controlado por mutex ao STDIN/STDOUT/STDERR extensível à outros recursos.
- DEBUG_MODE was, finally, normalized.
- 2017-10-21: RELEASE 1.0.8
- Better host identification for the html clients.
- Updating the html clients for the Datasources new identifiers.
- New Datasources. :-)
- Fetching logo (that "L" thingy) from http//lisias.net to reuse the "seasons" mechanism.
- Including the array "len" on the
.txt Responses (to easy the pain from the poor bastard that would need to debug this thing), and some small Serialization enhancements.
- "Stubborn" FTP connections. Now the data mining process is resilient.
- New Datasources
- Pigwa Net
- Apple II Documentation Project
- 2017-09-14: RELEASE 1.0.7
- Small front-end enhancement.
- Stupid mistake fixed while registering a service into the Confederation.
- ftp.arnold.c64.org is back online! :)
- Resurrecting the respective micro service.
- Link to the issue.
- Reenabling Metalab's PDP Search Engine.
- Yet better Exception handling on the Miners.
- Splitting up (finally!) the Mining into two phases: "Producer" and "Consumer".
- Consumers (specifically
file.search.WS) now can just load the datasets, ignoring the
datasource for the faulty ones.
- Producers can be sporadically executed (crontab!) on the server box (or even another one, exporting the files using NFS), saving your little server appliance from burning silicon due munching the huge data processed by the Miners.
- A more powerful auxiliar box, not necessarily a direct member of the Confederation, can take the job! We don't have to fry our poor Raspberries Pi anymore! :-)
- Some Known Issues were solved. :-)
- Sadly, am unwelcome but needed change: ftp.arnold.c64.org is dead. :'(
- One (out of date) mirror was found, and a copy is being fired up in ftp://ftp.service.retro.lisias.net while better alternatives are not found.
- The existing backup in Archive Org is yet older (2014/May).
- 2017-05-22: RELEASE 1.0.6c
- Some very stupid mistakes were corrected (yep, I need to be better)
- Added Web Players for:
- AmigaScne (but not activated yet by lack of computing resources)
- Funet's Amiga
- Refactoring FileData to improve maintenance and resilience.
- Better exceptions management on repository updates.
- 2017-04-21: RELEASE 1.0.6b
metalab.pdp Datasource was deployed with a very stupid mistake =/ . Fixed
Magazines handling on the
WOS Datasource. The
Description field (from
FileData) holds the number os pages/files per issue/book, and also the total files size.
- (finally) making everything thread-safe.
- Loading (and parsing when needed) of the Search Engine Datasources are done in parallel (4, to be exact) now.
FileData format, with new data items and
Description field compression.
- Allowing the supported compression on HTTP Request to be configurable (the RPi's were getting Memory Faults when handling ZX and LZMA). Now you can allow only the compression mechanism your appliance have installed (or can handle).
- Validating the
FileData received by WebRadio
- Only known files from known DataSources are accepted.
- The same criteria used to sort music from data files on the front end was also implemented on the back-end.
- Yes, I'm paranoid! :-)
- 2017-03-26: RELEASE 1.0.6
- More datasources for the Search Engine!
- Padua C64
- Amiga Scne
- Metalab's PDP directory
- Sam Coupé
- More datasources integrations to the Internet Radio:
- NVG's Sounds
- back-end enhancements
- Better and more versatile caching for repository data.
- Description "sanitizing" on repositories where it's usual to filler up the
.diz file with whistles and bells (inheritage from the BBS times!)
- Data format customization (and better granularity) on the Search Engine responses
- File last update date (without time)
- See documentation for details.
- Some more caring to the WebRadio
- Metadata adjustments, easing the client's life (in special, the console one)
- Proxy adjusts, to allow metadata deserialization.
- WebRadio Console Client!
- Discover by itself the current broadcasting stations, and choose one when the requested one is not found.
- A nice
asyncio and threading job made in Python!
- 2017-02-28: RELEASE 1.0.5b
- UI ajustments (SDK)
- A dedicated Internet Radio page.
- Finally making that fscking Dialog Boxes work right on Mozilla Firefox (that are dragging their feet to correctly implement the HTML5
- 2017-02-10: RELEASE 1.0.5
- WebRadio (heavy) refactoring
- Multiple sources for a given Station
- Better organization for the datasources/entities
- AmigaMod Station
- Previously, ModLand
- Needed to allow grab AmiNet files too.
fsid concept (file server ID) for the
FileData objects (the one that describes a file in some repository anywhere in the World)
- Allow multiple sources for the WebRadio
- Will allow a Mirroring Micro Service!
- Local clients (SDK)
- More examples (JSON, Brute Force)Força Bruta)
- Better organization of the existing ones.
- Front-End HTML (SDK)
- AmiNet now is also a feed for the WebRadio/AmigaMod!
- Solving cross dependencies that were missed
- Reuse promotion
- Better functionality decoupling
- modland.html refactoring
- Adapting it to the "new" WebRadio concept
- Request for the Playlist contents! :)
- 2017-01-31: RELEASE 1.0.4
- First command line client (in Python) for the Confederation!
- A very simple program that search for a file in every known Datasource, and consolidating the results.
- 2017-01-30: RELEASE 1.0.3
- "Full duplex" serialization.
- Transparent deserialization support (or almost) for every handled format (and compression) used by the requests (except, obviously, .TXT)
- Now the Micro Services can talk to each other, if needed.
- Partial listing for the net.lisias.retro.file.search.WS entities
- Very important for the Thin Clients
- Responses compression support
- A DynDNS tool for AWS users
- Ideally this would be on my Router, but the Lua environment currently used by that thing needs updating and the OpenWRT that will fix that is not available yet.
- Automatic Deserialization support implemented
- Very important for the SDK, and for new Services being builded using the existent ones.
- New FTP Search Engine Datasources
- Yet better HTML on front-ends
- Adjusts on the "dynamic" CSS mechanism
- Adding response compression (Accept-Encoding : xr, lzma, bzip2, gzip e deflate)
- adding parameters for
list (on file search)
prefix : return only words starting with the given substring
count : sort ascending and return the first words only
- Better HTML on front-ends
- Managed presentation format for Landscape and Portrait devices.
- Specialized versions for thin and "thick" clients.
- 2016-12-11: RELEASE 1.0.2
- Persisting parsed data using Pickle files, allowing (much) faster loading times as long the cache is valid.
- Binary format "pickle" for WebServices responses. Great for python clients.
- Annunciators were normalized. Better
atexit handling by
- Better Datasources configuration for the Search Engines
- Front End for the Database Query Providers.
- A dedicated physical subnet to the Confederation.
- Front ends HTML for the new Datasources
- Garbo (MS-DOS & Turbo Pascal)
- Hobbes (OS/2)
- Hornet (PC Demos)
- TV-Dog (Tandy 1000)
- x2ftp (Demos & MS-DOS)
- New Datasources for the Search Engines
- Garbo (MS-DOS & Turbo Pascal)
- Hobbes (OS/2)
- Hornet (PC Demos)
- TV-Dog (Tandy 1000)
- x2ftp (Demos & MS-DOS)
- 2016-11-18: RELEASE 1.0 FIRST_BLOOD
- Project Refactoring:
- PUB - public repository aimed to peopled interested in hosting Providers.
- SDK - public repository to Provider developers
- DEV - private repository where the kernel is developed.
- 2016-11-13 :
- Persistence: Proxy will save its internal states at shutdown, and tries to rebuild the Confederation at start-up.
- Postgresql. A Provider to serving controlled queries to a Postgresql database.
- 2016-11-09 :
- Persistence: The Providers now save theirs internal states on shutdown, allowing recovering at start-up.
- 2016-11-06 :
- Revising singleton support on proxy.WS.
- 2016-11-05 :
- Confederation: Singleton Entities
- A Singleton [Datasource, Entity] tuple can only be hosted by ONE Provider in the while Confederated Unit (and probably the while Confederation - this matter is still research).
- proxy.ws was rewritten to cope with Singleton Entities.
- CORS was implemented on Providers and Proxy.
- mpd.WS was refactored to provide metadata describing its Singleton Services, as well URLs for accessing its Channels (specialized entities)
- The Modland HTML Client was instrumented to enable client side "programming" for the Web Radio playlist.
- Previous, Next, Play, Delete - these actions costs "credits", renewed periodically.
- Requests for the proxy.WS are done by JSONP where possible, and fallbacks to XmlHttpRequest/CORS when needed.
- 2016-11-02 :
- proxy.WS : Services pulverization
- mpd.WS : alpha version to Remote Media Player
- 2016-10-28 :
- proxy.WS (Service Providers Federation)
- Registering and Unregistering protocols.
- 2016-10-25 :
- Handling localStorage problems on the HTML5 clients.
- Indexing AmiNet (Amiga)
- Indexing WhTech (TI99)
- Screw the cycles! :-D
- 2016-10-21 :
- Finishing (again) this cycle. :-)
- 2016-10-20 :
- Quick refactoring to promote extensibility.
- "Federation" metadata support
- 2016-10-19 :
- REST interface goes gold.
- P/R :-)
- Finishing this development cycle.
- 2016-10-18 :
- Indexing the MSX Archive.
- Facing some shi... I mean... unhappy decisions I made. =P
- HTML clients revamp.
- Performance tests, Profiling e Benchmarking.
- 2016-10-17 :
- Indexing a WoS mirror.
- Stability and performance tests.
- 2016-10-16 :
- http2.bash - Definitive Version.
- New pages for P/R
- net.lisias.retro.file.search.WS (how in hell I name this?) .
- Search engine for FTP indexing Asimov.
- 2016-10-13 :
- New, more advanced, implementation.
- 2016-10-12 :
- minimalist HTTP Server written in BASH