www.retrosoftware.co.uk :: View topic - Combining efforts on PC tools?

Intergrated IDE for C, ASM, and BASIC. (pretty far on with this)
Commandline tool for rolling SSD's
Raw text to BASIC tokenizer to produce code that will run on a Beeb
C compiler based on cc65 (trying to update Dominic Beesley's version to the latest code base)

No promises because I'm almost too busy to think right now, but if you're able to keep the logically platform neutral bits sufficiently separate, I expect it'd be relatively easy for someone like me to knock up a Mac sister IDE. You know, explicitly fashioned after the Windows original but altered to fit the different OS conventions.

...
I'm no Linux expert, but I suspect (correct me if I'm wrong) that it would not allow such total control over another application.

I'm no Linux expert, but I suspect (correct me if I'm wrong) that it would not allow such total control over another application.

Didn't really have any time on holiday or a lot of time last week, but I've been looking into it, and it looks like a miniature web server is relatively easy to hook up (see, e.g. http://www.ibm.com/developerworks/systems/library/es-nweb.html — "a simple Web server with only 200 lines of C source code"), so I propose that for cross-platform compatibility and to make things reasonably self-explanatory, a spec be based on the emulator running a web server, probably (or optionally?) on a non-standard port (in case the user wishes to run a real web server too), responding to URL-based queries with JSON or XML (as long as we pick one, does it matter which?) responses.

My feeling is that this will add hopefully only a few hundred lines maximum for the communication layer to most emulators, which tend to be C or C++ based, and slot nicely with the various bits of framework provided natively by C#, Objective-C, Java, Delphi, etc, since it'll just be asynchronous HTTP fetches with the usual language stuff for chucking results back into the main runloop (or whatever your language uses) once they're ready.

The debugger will be able to query and set machine state (which is defined broadly to include registers, memory contents, current screen display, raster position, etc — will possibly need someone from a BBC background to fill me in on various bits of state that aren't in the memory map, I guess stuff like the current screen display that is a record of events already occurred can be queried but not directly set), set conditional breakpoints and request that the emulator run according to a certain principle (eg, forever, for one frame, for 234 cycles, for 16 opcodes) or pause as soon as it is able.

Is this an acceptable way forward? If so, I'll try to produce a draft spec of some sort.

Why waste time using HTTP, why not make a new protocol? It's not exactly hard...

Quote:

Why waste time using HTTP, why not make a new protocol? It's not exactly hard...

For reasons including that:
[*] every modern framework already supports HTTP. It's therefore much less hard to use HTTP than any other protocol;
[*] there's also already quite a lot of documentation on HTTP, so for me and you maintaining that is quite a bit less hard;
[*] designing a new protocol is actually quite hard once you're doing anything non-trivial — that's why standards bodies take years thinking about these things.

The first point is possibly more wide reaching than you're allowing for. For example, it means that once an emulator is written that supports the HTTP control/debug stuff, a full IDE and debugger could be written in something abstract from the hardware and completely platform neutral, like HTML and Javascript.

I went my own way with UEF because I thought that a compact binary format that is easy to forward parse was correct and I was the only person involved in the ecosystem at the time. For communicating between multiple tools in multiple languages by multiple authors, I think building on what has already been implemented and doing as much as possible in a self-documenting manner (ie, communications in plaintext across an already implemented protocol) is smartest.

EDIT: actually, I think I've missed the thrust of your post. How would using HTTP waste time?

Well, you'd be setting up all the connections and transferring all the information, etc required to serve a web page, when in the end you just need a byte or two of info.

Author:	elaverick [ Wed Aug 26, 2009 1:00 am ]
Post subject:	Combining efforts on PC tools?
It sounds as though a few people are working on some similar PC tools at the moment (SSD format interpretation, BASIC compilers/tokenizers, my C compiler). Do we want to try and get together and share some of the info on this. I have a feeling that a few of us are working on some very similar projects. To get the ball rolling, I'm currently working on... Intergrated IDE for C, ASM, and BASIC. (pretty far on with this) Commandline tool for rolling SSD's Raw text to BASIC tokenizer to produce code that will run on a Beeb C compiler based on cc65 (trying to update Dominic Beesley's version to the latest code base)

Author:	MurrayCakaMuzer [ Wed Aug 26, 2009 8:29 am ]
Post subject:	Re: Combining efforts on PC tools?
elaverick wrote: Intergrated IDE for C, ASM, and BASIC. (pretty far on with this) Commandline tool for rolling SSD's Raw text to BASIC tokenizer to produce code that will run on a Beeb C compiler based on cc65 (trying to update Dominic Beesley's version to the latest code base) Multi-platform?

Author:	elaverick [ Wed Aug 26, 2009 3:04 pm ]
Post subject:	Re: Combining efforts on PC tools?
MurrayCakaMuzer wrote: Multi-platform? The IDE won't be, it's quite heavily reliant on WPF (I'm teaching myself WPF atm so I need some apps under my belt that use it). Most of the tools are C based atm tho so porting should be reasonably trivial (they don't do anything fancy, its just file format conversion for the most part).

Author:	ThomasHarte [ Mon Aug 31, 2009 12:21 am ]
Post subject:	Re: Combining efforts on PC tools?
No promises because I'm almost too busy to think right now, but if you're able to keep the logically platform neutral bits sufficiently separate, I expect it'd be relatively easy for someone like me to knock up a Mac sister IDE. You know, explicitly fashioned after the Windows original but altered to fit the different OS conventions. I don't actually have a sufficiently recent copy of Windows to run much .NET, but I have Windows 7 on order so that shouldn't be a problem for long.

Author:	elaverick [ Mon Aug 31, 2009 5:34 pm ]
Post subject:	Re: Combining efforts on PC tools?
ThomasHarte wrote: No promises because I'm almost too busy to think right now, but if you're able to keep the logically platform neutral bits sufficiently separate, I expect it'd be relatively easy for someone like me to knock up a Mac sister IDE. You know, explicitly fashioned after the Windows original but altered to fit the different OS conventions. That would be fantastic if you could. Just to clarify I know we already have Swift for ASM development, which I think is ace but oriented heavily towards experienced ASM coders. BIDE (Beeb IDE) is designed to be a complete development solution for BASIC and ASM, hopefully allowing users to knock up everything from quick BASIC test programs to full games. I'm aiming to put in a lot of helpers for common tasks (VDU 23 graphics plotting etc).

www.retrosoftware.co.uk http://www.retrosoftware.co.uk/forum/

Combining efforts on PC tools? http://www.retrosoftware.co.uk/forum/viewtopic.php?f=73&t=335	Page 1 of 2

Author:	ThomasHarte [ Sun Sep 06, 2009 4:08 pm ]
Post subject:	Re: Combining efforts on PC tools?
Actually, I'm being a bit overkeen on this but that's partly because I've intended to do some 6502 stuff for a while and I believe there to be no project management tools for OS X whatsoever — needlessly and offensively competing with Swift isn't a concern because even Swift isn't available. So, ummm, I've already started messing around. It's not going to be ready for a while, but I'm working on the idea that you put a bunch of .a65 and .bas files into the project, there's obviously a built-in editor for creating those, you hit build and/or run and those are assembled or tokenised into separate files on an SSD, an emulator is launched with the SSD inserted. All project settings are stored in an XML (self documentation and platform neutrality being beneficial if we're going to coordinate), and eventually the aim is to include a debugger to whatever extent it is possible, right now I think by TCP comms with an emulator that has a debugging server. Though that'll mean a whole bunch of complexity that'll need to be considered. I'm imagining compatible emulators have a fake write-only register (optionally) enabled somewhere in the system ROM space, code that expects to be debugged writes a special string and the source file name to that register, at which point the emulator twigs that it is expected to debug and starts listening on a socket. Subsequently it can be told to run for a certain number of cycles, a certain number of opcodes or indefinitely, and can be given PC and memory read/write trap addresses. Probably it'd also be helpful if there was some way to tell the emulator to run until a certain raster position and to display the screen up to that point only. Also I guess there'd need to be a way to query various bits of machine state. Then the IDE could collect various bits of information from the assembler/tokeniser and map back to let you step through your actual code. I can build all that into ElectrEm relatively easily (it's been set up for years to support debugging, it's just cross-platform user-interface issues that have prevented me exposing the functionality), then it should be able to host debugging sessions either here in OS X from my IDE or over on Windows from any other IDE using the same protocol. And hopefully someone would be able to add the same stuff to a BBC emulator so that any related IDE could connect to either type of target machine. Anyway, apart from the basic Cocoa stuff of a file list and an editor and saving/loading projects and files, I've really done nothing, so really I'm just thinking aloud. Hopefully my general thoughts are the same as yours have been?

Author:	elaverick [ Sun Sep 06, 2009 5:21 pm ]
Post subject:	Re: Combining efforts on PC tools?
Yes that sounds very much what I had in mind, I've not gone so far into the details of the debugging however. Another feature I'd been looking into is supporting Mark Haysman's Data Centre addon for direct deployment of SSDs over the USB link. The emu's are pretty fantastic at the moment, but they're not 100% so a direct testing solution would be quite cool too. I like the neutrality of XML for the project settings, and possibly extending that to the IDE settings as well. I've not got massively far on the tools front yet, short programs seem to tokenize fine but anything longer than a couple of lines falls over. Examining files generated by the tokeniser and the Beeb in a Hex editor, the PC Tokenised files appear to be a touch longer and are a few digits off for various symbols. I'm wondering if it's to do with CR/LF issues? However when I run the same file through ElectrEm it seems to tokenise perfectly... We'll have to get together shortly and compare designs as well, I'm working on a TDI in a similar vein to Visual Studio at the moment. I'll post some screenshots later.

Author:	SteveO [ Tue Sep 15, 2009 2:53 pm ]
Post subject:	Re: Combining efforts on PC tools?
Strangely enough, the latest (unreleased) version of Swift now has debugging support for BeebEm. In Swift you simply click on a source line in your code where you want a break point to be and when you launch the emulator it will stop at that point, highlight your line of code in your source and you can single step through your source as it executes on BeebEm. You can roll you mouse over memory or labels and the current values appear, much like any other high level debugger that people may have encountered. Massive downside is that it is heavily tied to BeebEm and not cross platform It's tied to BeebEm as I haven't requested any work from the author of that to make this work so the tricks I employ to get BeebEm to do what I want wouldn't work with another emulator even if they did support debugging. It's not complete yet, but was around 70% done about 8 weeks back or so when I last worked on it. If I can lend a hand on any of this future collaboration give me a shout and time permitting I'll help where I can.

Author:	ThomasHarte [ Sun Sep 20, 2009 9:11 pm ]
Post subject:	Re: Combining efforts on PC tools?
Any chance you could outline exactly what you've done with BeebEm? Apart from wanting to see if I could add it to ElectrEm, it could be instructive.

Author:	SteveO [ Sun Sep 20, 2009 9:43 pm ]
Post subject:	Re: Combining efforts on PC tools?
Briefly (well as brief as I can !), on windows if you can get the handle (a pointer) to the application (not too hard) you can then get the handles to all the visual components in a window etc. etc. First thing I do when launching BeebEm is supply the debugging addresses on the command line (normal BeebEm thing, not sure when it was implemented). I quickly get the handle to the debugging window and (currently) hide it, but thinking of making it viewable on demand/ optional. This gives me a debugging window already set up with my breakpoints. I've got the handles to the main code listing area, the watches area, the command entry field and the execute button. With all these handles I can directly manipulate the debugger. Swift watches the main code list text area for the address being executed and can match that with a line in the source code (as it knows where all the source code is going to be because of the label/value facility it has. To step through the code all swift does is fill the command field with the word "next" and generates a click on the "execute" button and hey presto ! To get values when the mouse rolls over vars etc. I make use of BeebEms debugger memory dump/ watches facility. Swift knows all memory locations for all vars in your source (as it analyses it on every assembly). So it can pass memory locations to BeebEm in a similar fashion already outlined and get the values back, again by inspecting the BeebEm debugger window. Swift also gets all registers information/ program counter etc. information from BeebEm debugger indow and shows the information itself in the same sort of way again. It all looks "magical" when the debugger window is hidden (which again Swift can do as it has it's window handle), as you have all the control from Swift and it's as if BeebEm is executing your source a line at a time. The variable/ memory location mouse roll over is particularly pleasing and makes debugging sweet. I'm no Linux expert, but I suspect (correct me if I'm wrong) that it would not allow such total control over another application. But of course if we could persuade some built in comms from BeebEm author then that would make it do-able on other systems. I just don't like to trouble people so I dived in with this solution.

Author:	ThomasHarte [ Sun Sep 20, 2009 11:09 pm ]
Post subject:	Re: Combining efforts on PC tools?
Oh, neat hack (in the uninflected, traditional meaning)! Obviously not something I can implement on ElectrEm and get Swift compatibility. SteveO wrote: ... I'm no Linux expert, but I suspect (correct me if I'm wrong) that it would not allow such total control over another application. I've no idea about Linux either, but it sounds doubtful given how stone aged the X protocol is. You should be able to do the same thing on OS X though (probably without hiding the debug window), since the built in scripting language can perform commands like "tell button "button" of window "window" to perform action" and I don't believe it has elevated status. Of course, the correct thing to do is to offer the various bits as explicitly scriptable actions, allowing any other app to trigger them, but that's definitely not portable. I have a little holiday from work this week, I might sketch out a little debugging protocol. I'll post anything I come up with for discussion when I get back...

Author:	MurrayCakaMuzer [ Sat Oct 03, 2009 9:35 pm ]
Post subject:	Re: Combining efforts on PC tools?
SteveO wrote: I'm no Linux expert, but I suspect (correct me if I'm wrong) that it would not allow such total control over another application. You'd obviously need to modify beebem, but there are plenty of inter-process communication tools - the most common one nowadays is d-bus. This would probably allow what you want.

Author:	ThomasHarte [ Mon Oct 05, 2009 12:15 pm ]
Post subject:	Re: Combining efforts on PC tools?
Didn't really have any time on holiday or a lot of time last week, but I've been looking into it, and it looks like a miniature web server is relatively easy to hook up (see, e.g. http://www.ibm.com/developerworks/systems/library/es-nweb.html — "a simple Web server with only 200 lines of C source code"), so I propose that for cross-platform compatibility and to make things reasonably self-explanatory, a spec be based on the emulator running a web server, probably (or optionally?) on a non-standard port (in case the user wishes to run a real web server too), responding to URL-based queries with JSON or XML (as long as we pick one, does it matter which?) responses. My feeling is that this will add hopefully only a few hundred lines maximum for the communication layer to most emulators, which tend to be C or C++ based, and slot nicely with the various bits of framework provided natively by C#, Objective-C, Java, Delphi, etc, since it'll just be asynchronous HTTP fetches with the usual language stuff for chucking results back into the main runloop (or whatever your language uses) once they're ready. The debugger will be able to query and set machine state (which is defined broadly to include registers, memory contents, current screen display, raster position, etc — will possibly need someone from a BBC background to fill me in on various bits of state that aren't in the memory map, I guess stuff like the current screen display that is a record of events already occurred can be queried but not directly set), set conditional breakpoints and request that the emulator run according to a certain principle (eg, forever, for one frame, for 234 cycles, for 16 opcodes) or pause as soon as it is able. Is this an acceptable way forward? If so, I'll try to produce a draft spec of some sort.

Author:	SteveO [ Mon Oct 05, 2009 1:06 pm ]
Post subject:	Re: Combining efforts on PC tools?
Sounds reasonable to me. Definitely need to have the http port configurable as I myself run several web services on different ports already ! I'd add in to the specs the ability to control the emulator as well, in the sense of "Issue a reset", "Perform Boot" etc. And if we add in the facility to send over a disk image to the emulator for a "Load remote disk option" people could run the emulator on a physically different PC. I'll put on hold my debugger for now, as it would be pointless to waste effort only to replace it with something else. However if it looks like it's going to take months and months for the emulators to support this I'll probably finish my current version anyway, just so I've got something to help me with my own coding, and add in support for this when it comes along.

Page 1 of 2	All times are UTC [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/

Author:	MurrayCakaMuzer [ Tue Oct 06, 2009 6:17 pm ]
Post subject:	Re: Combining efforts on PC tools?
ThomasHarte wrote: Didn't really have any time on holiday or a lot of time last week, but I've been looking into it, and it looks like a miniature web server is relatively easy to hook up (see, e.g. http://www.ibm.com/developerworks/systems/library/es-nweb.html — "a simple Web server with only 200 lines of C source code"), so I propose that for cross-platform compatibility and to make things reasonably self-explanatory, a spec be based on the emulator running a web server, probably (or optionally?) on a non-standard port (in case the user wishes to run a real web server too), responding to URL-based queries with JSON or XML (as long as we pick one, does it matter which?) responses. My feeling is that this will add hopefully only a few hundred lines maximum for the communication layer to most emulators, which tend to be C or C++ based, and slot nicely with the various bits of framework provided natively by C#, Objective-C, Java, Delphi, etc, since it'll just be asynchronous HTTP fetches with the usual language stuff for chucking results back into the main runloop (or whatever your language uses) once they're ready. The debugger will be able to query and set machine state (which is defined broadly to include registers, memory contents, current screen display, raster position, etc — will possibly need someone from a BBC background to fill me in on various bits of state that aren't in the memory map, I guess stuff like the current screen display that is a record of events already occurred can be queried but not directly set), set conditional breakpoints and request that the emulator run according to a certain principle (eg, forever, for one frame, for 234 cycles, for 16 opcodes) or pause as soon as it is able. Is this an acceptable way forward? If so, I'll try to produce a draft spec of some sort. Why waste time using HTTP, why not make a new protocol? It's not exactly hard...

Author:	ThomasHarte [ Tue Oct 06, 2009 8:12 pm ]
Post subject:	Re: Combining efforts on PC tools?
Quote: Why waste time using HTTP, why not make a new protocol? It's not exactly hard... For reasons including that: [] every modern framework already supports HTTP. It's therefore much less hard to use HTTP than any other protocol; [] there's also already quite a lot of documentation on HTTP, so for me and you maintaining that is quite a bit less hard; [*] designing a new protocol is actually quite hard once you're doing anything non-trivial — that's why standards bodies take years thinking about these things. The first point is possibly more wide reaching than you're allowing for. For example, it means that once an emulator is written that supports the HTTP control/debug stuff, a full IDE and debugger could be written in something abstract from the hardware and completely platform neutral, like HTML and Javascript. I went my own way with UEF because I thought that a compact binary format that is easy to forward parse was correct and I was the only person involved in the ecosystem at the time. For communicating between multiple tools in multiple languages by multiple authors, I think building on what has already been implemented and doing as much as possible in a self-documenting manner (ie, communications in plaintext across an already implemented protocol) is smartest. EDIT: actually, I think I've missed the thrust of your post. How would using HTTP waste time?

Author:	MurrayCakaMuzer [ Tue Oct 06, 2009 8:16 pm ]
Post subject:	Re: Combining efforts on PC tools?
ThomasHarte wrote: Quote: Why waste time using HTTP, why not make a new protocol? It's not exactly hard... For reasons including that: [] every modern framework already supports HTTP. It's therefore much less hard to use HTTP than any other protocol; [] there's also already quite a lot of documentation on HTTP, so for me and you maintaining that is quite a bit less hard; [*] designing a new protocol is actually quite hard once you're doing anything non-trivial — that's why standards bodies take years thinking about these things. The first point is possibly more wide reaching than you're allowing for. For example, it means that once an emulator is written that supports the HTTP control/debug stuff, a full IDE and debugger could be written in something abstract from the hardware and completely platform neutral, like HTML and Javascript. I went my own way with UEF because I thought that a compact binary format that is easy to forward parse was correct and I was the only person involved in the ecosystem at the time. For communicating between multiple tools in multiple languages by multiple authors, I think building on what has already been implemented and doing as much as possible in a self-documenting manner (ie, communications in plaintext across an already implemented protocol) is smartest. EDIT: actually, I think I've missed the thrust of your post. How would using HTTP waste time? Well, you'd be setting up all the connections and transferring all the information, etc required to serve a web page, when in the end you just need a byte or two of info.

Author:	ThomasHarte [ Tue Oct 06, 2009 8:25 pm ]
Post subject:	Re: Combining efforts on PC tools?
Quote: Well, you'd be setting up all the connections and transferring all the information, etc required to serve a web page, when in the end you just need a byte or two of info. I suppose maybe it's bad from a machine efficiency angle, most of the time (obviously it'll be more than a byte or two if the command is to transmit a disk image or request a chunk of memory), but from a coding efficiency point of view it'll be just a few lines rather than having to manually implement a new protocol, which is especially helpful at the sort of level you want to be working when writing an IDE. And I doubt it'll be any sort of bottleneck.

Author:	SteveO [ Tue Oct 06, 2009 9:16 pm ]
Post subject:	Re: Combining efforts on PC tools?
Personally I wouldn't add to the time involved by creating a new protocol. The HTTP one suggested may not be the most optimal for the few bytes that will often be exchanged on debugging but it will be more than fast enough for the human interaction required and quicker to implement as we're not having to create a new underlying protocol as well. For me anything that already exists and can perform to design requirements is good to go with If there are distinct advantages to a new protocol I can change my mind

Author:	ThomasHarte [ Tue Oct 06, 2009 9:30 pm ]
Post subject:	Re: Combining efforts on PC tools?
The main advantage I've been able to come up with for a new protocol would be that it might be easier to implement as a hardware box to connect to a real BBC. But I think my feeling is that probably the correct thing would be to just run an HTTP server on a PC (or Arc or whatever) connected to the box, with exactly how those two communicate to be decided by anyone who might like to make such a box. And even then it'd probably be an expediency decision, since there are TCP/IP stacks and HTTP servers that run on machines like the C64, so required processing resources obviously aren't too massive. Actually being a hardware monitor would almost certainly be a much bigger problem.