It is currently Mon Oct 20, 2014 4:35 pm

All times are UTC [ DST ]




Post new topic Reply to topic  [ 27 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Sun Feb 27, 2011 9:08 pm 
Offline
Site Admin
User avatar
 Profile

Joined: Wed Dec 19, 2007 10:46 pm
Posts: 779
Hey, all.

This is sort of related to Dave's recent feature request for BeebAsm - a copy command.

I had a similar need - I wanted a cross-platform tool to be able to add a file directly to an SSD, but I also wanted to be able to specify that it was a BASIC text file and that it should be tokenised before it was saved. I initially suggested it to Rich as a new feature for BeebAsm or as a companion binary executable, but Rich wasn't really keen on including that sort of functionality in BeebAsm itself.

I take the point and, the more I think about it, the more I think it would be better to have a standalone cross-platform SSD manipulation tool which basically does the same sort of disc image manipulation as OmniDisk without the disc reading, but that can be compiled natively on non-Windows platforms. (@Dave, if you're using Windows, I think you should be able to use OmniDisk to add a file as you wanted).

Starting with my use case of wanting to be able to add a file to an ssd with the option of tokenising it, I planned out an add2ssd binary which I've been hacking away on. Thomas Harte has provided us with the guts of a BASIC tokenising routine on the wiki, originally taken from ElectrEm, and of course BeebAsm ships with the add functionality, so in theory it should be trivial to plug the two together. Of course, my C++ sucks so badly, I still haven't got it working even with Rich's patient pointers. It compiles on both Linux and MinGW, but under the former it crashes out immediately with a Segmentation Fault. Under MinGW, it appears to sort of work - if the file is renamed to DEMO (it can't handle the $.DEMO.bas yet) it adds the DEMO BASIC file and to a newly generated SSD, but when I open it up the file on the new disc image is of size 0. :/

The archive includes some documentation on how the tool is intended to work in about.txt, and there are some relevant TODOs commented in the code for stuff which still needs fleshing out.

Since I've been badly hacking away at this, Dave has posted some SSD manipulation C code of his own here that I haven't yet had a chance to look at. In addition to that, I've also discovered that Dominic Beesley released a GPL v2+ command line C app well over five years ago, DFS v0.3 which - from the description - sounds exactly like what I think is needed, bar the BASIC tokenising use-case. Unfortunately, I've been unable to get that to compile with my current MinGW and Linux environments so have no idea how much functionality it contains.

Anyway, as Dave has raised the fact that he was looking at a similar use-case, I thought it best that I posted my progress (or lack thereof) here in public and also stop bugging Rich in private over it. I'm preparing for an absence next week, which means I'm unlikely to touch this again until I get back, in nearly a month.

If anyone wants to play with it or further develop it, feel free. It's been painfully slow so far. I will endeavour to finish this add2ssd utility as documented at least when I get back, but I think I'll struggle to take it any further. Ideally, it'd be great to expand the tool to be a full ssd manipulation tool, but I think that's a bit wishful thinking from my point of view, at this point.

Sam.


Attachments:
add2ssd-0.1.zip [39.47 KiB]
Downloaded 17 times
Top
 
PostPosted: Sun Feb 27, 2011 11:23 pm 
Offline
 Profile

Joined: Sat Sep 04, 2010 5:28 pm
Posts: 92
My disc utilities (linked in the other thread) used to be available about 10 years back from the BBC lives (though they link to an old demon account that I haven't had active for nearly 10 years)!

The original concept was that I'd have multiple library files which had the code for different flavours of disc image (BBC, D64, Atari, PCW) and that the front-end utilities would access them as if they were disc volumes. As I was a big Unix sysadmin at the time, they were xxxls, xxxcp and xxxmk to list, copy and make.

I then tried to apply them to fuse which I sort of failed at - mainly as fuse is very badly documented and seems to change spec every year :-)

I'll have a look at your code and see what I can do. It may be an idea to produce a general library file that will read/write/manipulate DFS/ADFS images so that other utilities like Swift/BeebASM/Sprite Designer etc can use them.

Unfortunately I'm only a Windows user at the moment - my better half complained about all the computers I had, so I kept only the games machine and my Sheevaplug; which then blew up! So until I can get around to setting up a VM, I'm Windows only.


Top
 
PostPosted: Mon Feb 28, 2011 12:44 am 
Offline
Site Admin
User avatar
 Profile

Joined: Wed Dec 19, 2007 10:46 pm
Posts: 779
Interesting concept - tho isn't FUSE fairly *nix specific? It sounds like you've turned around entirely, if you're now Win32-only. :)

I don't really know much about how library files work under C++, but I think that's how the code is currently set out - it basically re-uses a direct copy of the discimage.cpp code from BeebAsm with no changes at all atm (except in the comments to say where it came from). If you're proposing we extend that file to cover more than just adding files, but also removing them, reading them etc. then that was pretty much what I envisaged when I started, albeit it would take me decades lol. The reason I went to Rich first with the feature request was that I thought it would be good to keep that library maintained in one place. Though I do now think he's right - BeebAsm isn't the best place for it, it really warrants a full project of it's own which is what I started hacking away at here. The command line app can be used in build processes etc. (maybe someone could even build a GUI front-end around it, an open source DFS explorer) whilst the library file itself could be taken and re-used as you suggest to give ppl a headstart in their own projects. Of course, the name add2ssd might need revisiting, if the scope grows to more than adding and perhaps even more than just DFS!

If the program compiles under MinGW on Windows, then there's a pretty good chance it should compile on Linux and installing MinGW isn't too tricky. That's what I used to compile the current version (tho, as I say it doesn't seem to work properly yet). I think trying to keep it cross-platform is a worthy goal, not least because I'm trying to build this tool for use in my Linux environment! But also I see it as being like BeebAsm - one of those "core" tools that would be really handy for everyone, whatever platform they're using.

Running Linux under VirtualBox is a pretty easy combination too - happy to give you pointers if you're new to virtualisation. Though I'm sure as an old skool *nix administrator, you'd pick it up fast even if you haven't looked at it before. (Can download full VMs direct from virtualboxes.org, if you don't want to bother going through a full install of Linux).

Anyway, I think everything I had in my head is now either in that source code bundle or described here. When I get back in a few weeks time, I'll continue plugging away trying to get the add2ssd binary finished.

Sam.

P.S. For completeness, it's probably also worth remembering that I'm sure the open source emus (BeebEm, B-Em etc. must have disc handling code that could be re-purposed, as they're both GPL as well).


Top
 
PostPosted: Fri Mar 04, 2011 6:01 pm 
Offline
 Profile

Joined: Sat Sep 04, 2010 5:28 pm
Posts: 92
Samwise wrote:
Interesting concept - tho isn't FUSE fairly *nix specific? It sounds like you've turned around entirely, if you're now Win32-only. :)


I've generally had a Linux partition or separate box since the 90s; but recently had to shrink my number of machines. As I play a lot of modern games, I had to keep my Windows box. But, there's always VMware.

I tried to use fuse as I wanted to have a similar frontend to ImageFS on RISC OS. Though I've found doing any of these sort of things on Linux very tiresome: the APIs change, the documentation sucks and the example don't work.

Samwise wrote:
If you're proposing we extend that file to cover more than just adding files, but also removing them, reading them etc. then that was pretty much what I envisaged when I started, albeit it would take me decades lol.


That's pretty much it; plus attempt bindings for multiple libraries.

So what I propose is that we attempt an object orientated design with a disk image/class, with the basic objects being files, directories and disk images.

So one would instantiate a new disk image (either blank or read from a file), then pull a list of directories/files from it and add. The back end should handle the fine details.


Top
 
PostPosted: Fri Mar 04, 2011 11:30 pm 
Offline
Site Admin
User avatar
 Profile

Joined: Wed Dec 19, 2007 10:46 pm
Posts: 779
Even more recent interest. ;)

I'm definitely on-board with the idea, though how much practical help I can be towards the end goal is another matter. I do have a C++ coding course coming up at work next month, but I wouldn't expect miracles. :) I can certainly do testing and if the project's open source (which I think is half the point), I can set up a project in the RS mercurial repository, do documentation etc.

That all said, I'm going to be off for a few weeks so can't do anything till I get back. I'll check in when I return and we'll see what's what.

If the scope is bigger than just adding files and even more than DFS, we'd definitely need to revisit the name of the project tho. BeebMIM: Beeb Media Image Manager?

I dunno! lol

Sam.


Top
 
PostPosted: Sat Mar 05, 2011 11:00 am 
Offline
 Profile

Joined: Sat Sep 04, 2010 5:28 pm
Posts: 92
I did some messing around last night and I have the basic classes in place (I'm more of a C guy than a C++ guy, so its a learning experience for me). At the moment I'm just reading the catalogue into a class structure.

The basic plan is to have two classes for DFS:
  • Disc
  • File

Instantiating a new disc will either read it from an existing source or will create a spanky new one. Disc will give a couple of methods, probably next and start; which'll return a File entry with everything in.

So the theory will be that a program to just copy a disc (and do a compact) will be summat like:
Code:
Disc idisc ("jim.ssd");
Disc odisc ();
File tfile;
idisc.start()
while (tfile = idisc.next())
{
   // set track and sector to zero to automatically compact disc
   tfile.track=0;
   tfile.sector=0;
   odisc.add(tfile);
}
odisc.write("fred.ssd");

Still not written these bits so they may change.


Top
 
PostPosted: Mon Jun 06, 2011 2:01 am 
Offline
User avatar
 WWW  Profile

Joined: Mon Mar 15, 2010 2:28 pm
Posts: 42
Location: Minehead, Somerset (UK)
I'm catching up with old threads having been out of it (not literally!) for months now. I'm not likely to be 'back into it' at the moment either, but I can probably add something to this...

Last year, or maybe the year before, I wrote some of this stuff for the Mac. The code's a bit hacky and probably very badly put together as I'm crap at visualising and designing objects prior to actually coding so I have a tendency to dive in and make it up as I go along (very, very bad!)

BUT, having said all that, I had all the code in place to READ SSDs and DSDs and was about to move onto writing them. Problem was, I achieved my goal, which was an OS X 'Quick Look' plugin for Mac and got absolutely zero interest whatsoever so never did any more to it. Basically, my quick look plugin works and gives me exactly what I wanted so it was basically job done for me.

Here's a reminder of what it was: http://screencast.com/t/MWU4NWVhZT
Although after that the output was themed to look a lot nicer with the BeebEm logo (with permission) etc.

The thread on STH was here: http://www.stardot.org.uk/forums/viewto ... =12&t=3020

Thing is, I wrote it in an isolated, re-usable form. All the 'QuickLook Plugin' stuff is in one project and all the meat and two veg of it is in 'Acorn Disk Tools for Mac'.

Dunno if any of it would be of any interest to anyone? It's written in Objective C using Cocoa frameworks, obviously since it target's OS X but at the end of the day, it ought to be pretty obvious to any C coder what's going on.

Like I say, zero interest from anyone meant I ended up just keeping it to myself.


Top
 
PostPosted: Mon Jun 06, 2011 11:04 am 
Offline
 Profile

Joined: Sat Sep 04, 2010 5:28 pm
Posts: 92
I got bored after rewriting my previous code into objects just to dump the catalogue...

The problem is that there's a selection of tools which do it through GUI-fication, so there's not much call for new utilities.

I suppose something that would be useful would be a selection of libraries to allow different BBC file and disk manipulation all nicely bound for different programming languages, so that people can mangle stuff as they see fit in whatever language they want, this could include stuff like:
  • DFS disc image manipulator
  • ADFS disc image manipulator
  • BASIC tokeniser/detokeniser
  • Openup file reader (I have one which I wrote a while back)
  • BBC sprite/screen -> bitmap

I personally would like to get stuff to read all the RISC OS file formats too (draw, sprite, outline fonts, impression etc). But, I'm put off by the lack of interest which always occurs on these.


Top
 
PostPosted: Mon Aug 08, 2011 3:05 am 
Offline
 Profile

Joined: Sat Aug 22, 2009 7:45 pm
Posts: 34
tautology wrote:
[*]BBC sprite/screen -> bitmap

BBCtoBMP over on mdfs.net.


Top
 
PostPosted: Mon Aug 22, 2011 11:01 pm 
Offline
User avatar
 WWW  Profile

Joined: Wed Jan 26, 2011 2:35 am
Posts: 139
tautology wrote:
I suppose something that would be useful would be a selection of libraries to allow different BBC file and disk manipulation all nicely bound for different programming languages, so that people can mangle stuff as they see fit in whatever language they want, this could include stuff like:
  • DFS disc image manipulator
  • ADFS disc image manipulator


Unfortunately, it's read-only, but a start: python-adfs. I also wrote UEFtrans to work with UEF files, which is quite handy when testing things with emulators.

tautology wrote:
  • BASIC tokeniser/detokeniser
  • Openup file reader (I have one which I wrote a while back)
  • BBC sprite/screen -> bitmap

  • I personally would like to get stuff to read all the RISC OS file formats too (draw, sprite, outline fonts, impression etc). But, I'm put off by the lack of interest which always occurs on these.


    I'm interested! I've been slowly writing code for the last ten years or so:


    The regulars in the RISC OS scene never saw the point of this stuff, but I didn't write it for them. The idea is to at least have a chance of reading files in the future and maybe even convert them to other formats.


    Top
     
    PostPosted: Fri Aug 26, 2011 8:49 pm 
    Offline
     Profile

    Joined: Sat Sep 04, 2010 5:28 pm
    Posts: 92
    I've got a drawfile one that I wrote a while back as well (attached). I posted it to Stardot and only had one person show any interest (the attached version is more up to date than the one on Stardot).

    I choose to convert draw to SVG, as all modern browsers support it (as of IE9) and it is actually quite a close map to the internals of the draw file.

    I've seen your Impression page in the past and keep on meaning to use it. I think we may try and team up our reverse engineering experience!

    Samwise; is it worth setting up an informal RS project for conversion tools for the Beeb and RISC OS file formats?

    We can easily convert some of the standard files, such as:
    Sprite -> PNG
    Draw -> SVG
    Font -> OTF
    Impression/Wordwise/View -> ODT

    ad nauseam.


    Attachments:
    Draw files.zip [12.6 KiB]
    Downloaded 9 times
    Top
     
    PostPosted: Fri Aug 26, 2011 10:06 pm 
    Offline
    User avatar
     WWW  Profile

    Joined: Wed Jan 26, 2011 2:35 am
    Posts: 139
    tautology wrote:
    I've got a drawfile one that I wrote a while back as well (attached). I posted it to Stardot and only had one person show any interest (the attached version is more up to date than the one on Stardot).


    I misread that as Slashdot, initially, and wasn't really surprised! :lol:

    tautology wrote:
    I choose to convert draw to SVG, as all modern browsers support it (as of IE9) and it is actually quite a close map to the internals of the draw file.


    I left it open as to what to convert to. Things get more fun when you have to deal with sprites embedded in Drawfiles, though handling JPEGs is pretty straightforward.

    tautology wrote:
    I've seen your Impression page in the past and keep on meaning to use it. I think we may try and team up our reverse engineering experience!


    That would be cool. A central hub for that kind of work would be useful, too.


    Top
     
    PostPosted: Fri Aug 26, 2011 11:37 pm 
    Offline
     Profile

    Joined: Sat Sep 04, 2010 5:28 pm
    Posts: 92
    davidb"I misread that as Slashdot, initially, and wasn't really surprised! [/quote]
    I actually typed it as slashdot at first, then had to alter it. Damn you Samwise! :-)

    [quote="tautology wrote:
    I choose to convert draw to SVG, as all modern browsers support it (as of IE9) and it is actually quite a close map to the internals of the draw file.

    I left it open as to what to convert to. Things get more fun when you have to deal with sprites embedded in Drawfiles, though handling JPEGs is pretty straightforward.[/quote]
    The best way to do them is to do it in steps, i.e.:
    • RISC OS->Neutral
    • Neutral->other
    So the user could decide whether to have SVG/PS/PDF/WMF, or whatever. I'm just too lazy to do it properly :-)

    In terms of Sprites I got permission from my old partner in crime (Ian Jeffray) to steal his Sprite->PNG convertor. I never got around to integrating it into my Draw routines though...


    Top
     
    PostPosted: Sat Aug 27, 2011 11:29 pm 
    Offline
    Site Admin
    User avatar
     Profile

    Joined: Wed Dec 19, 2007 10:46 pm
    Posts: 779
    Sure, what kind of resources are we talking about?

    I can set up a mercurial repository if you want to share code. Or I (or anyone else) can setup a template wiki page off the Dev Tools page if you just want to start there?

    Let me know and I'll get started setting something up ...

    Sam.


    Top
     
    PostPosted: Mon Oct 03, 2011 9:36 pm 
    Offline
    User avatar
     WWW  Profile

    Joined: Wed Jan 26, 2011 2:35 am
    Posts: 139
    One thing that would be nice to start with is a repository for my ADFS libraries and tools which is currently hosted at BerliOS.

    https://developer.berlios.de/projects/python-adfs/

    That site is closing at the end of the year, so another place to host the code would be good. Right now, I'm thinking of using Bitbucket, but somewhere like Retro Software would be more relevant.


    Top
     
    PostPosted: Mon Oct 03, 2011 10:02 pm 
    Offline
    Site Admin
    User avatar
     Profile

    Joined: Wed Dec 19, 2007 10:46 pm
    Posts: 779
    David,

    I agree - and it fits in perfectly with our Dev Tools, I think. I'm not keen on trying to support a git repository as well as Mercurial, though. Would you be alright with another Mercurial project(s) for your tools, supplemented by a page(s) off the Dev Tools wiki page?

    Sam.


    Top
     
    PostPosted: Mon Oct 03, 2011 11:17 pm 
    Offline
    User avatar
     WWW  Profile

    Joined: Wed Jan 26, 2011 2:35 am
    Posts: 139
    Samwise wrote:
    I agree - and it fits in perfectly with our Dev Tools, I think. I'm not keen on trying to support a git repository as well as Mercurial, though. Would you be alright with another Mercurial project(s) for your tools, supplemented by a page(s) off the Dev Tools wiki page?


    It's a Mercurial repository already so I'm good to go. Sounds perfect!


    Top
     
    PostPosted: Mon Oct 03, 2011 11:32 pm 
    Offline
    Site Admin
    User avatar
     Profile

    Joined: Wed Dec 19, 2007 10:46 pm
    Posts: 779
    OK, do you just want it called python-adfs? Is it just the one project?

    Sam.


    Top
     
    PostPosted: Tue Oct 04, 2011 6:59 pm 
    Offline
    User avatar
     WWW  Profile

    Joined: Wed Jan 26, 2011 2:35 am
    Posts: 139
    Yes, that sounds good. It's just the one repository.


    Top
     
    PostPosted: Wed Oct 05, 2011 12:43 am 
    Offline
    Site Admin
    User avatar
     Profile

    Joined: Wed Dec 19, 2007 10:46 pm
    Posts: 779
    Here ya go:

    http://www.retrosoftware.co.uk/hg/python-adfs/

    Same login details as your other repository.

    Sam.


    Top
     
    Display posts from previous:  Sort by  
    Post new topic Reply to topic  [ 27 posts ]  Go to page 1, 2  Next

    All times are UTC [ DST ]


    Who is online

    Users browsing this forum: No registered users and 1 guest


    You cannot post new topics in this forum
    You cannot reply to topics in this forum
    You cannot edit your posts in this forum
    You cannot delete your posts in this forum
    You cannot post attachments in this forum

    Search for:
    Jump to:  
    cron