Embedded Jetty, Atmosphere & Jersey

Backstory
So, my final major project for my bachelors degree's is software in nature, though it still has it's roots in hardware.  My project is to build a compiler / translator for my own custom HDL titled Silk. The compiler (named Weave) will output a Verilog file for import into a standard synthesis tool. Written in a mix of Java & Scala it is hoped to eventually integrate with Eclipse.

14.7 PSI
Atmosphere is an abstraction layer for bidirectional communication in web applications. You are probably thinking, Weave is a compiler not a webapp... You would be correct.  However, with more and more moving to the web it would be extremely convenient to have a websocket api for the compiler. At this point I haven't exactly figured out what atmosphere gives me over using the Jetty API directly.  The documentation for atmosphere is non-existent at best and outdated / wrong at worst.  It seems to have a lot of hype / people swearing by it so I will give it a shot.
To get started however I spent the last few days getting it compiling and running.  At this point I now have an embedded jetty server and atmosphere accepting websocket connections. I am unsure as to what the protocol is and will begin looking into that next as well as setting up connecting to the server on the other side.
So without further ado...

The Code
So first off here is the code that I had when I wrote this blog post:
https://github.com/Deathbobomega/Weave/tree/9a77f3d2b688a906f8ecc2c5dcd68452fb272018
For those who are also interested in continuous integration here is the build server:
http://ci.codingwell.net/job/Weave/12/

I do believe I would have never figured this out (or would have required several back and forth posts on the mailing list) if it wasn't for Charles Brown. He posted some example code to github that I found extremely helpful.
https://github.com/charlesbrown/Jersey-Atmosphere-with-Embedded-Jetty

I however ended up chasing my tail for hours/days trying to figure out why I had the same problem he did:
http://markmail.org/message/6wxzztdwbgfgxmp3

But his solution only sort-of worked for me. This also made me wonder why the github code (which was newer) did not include this "fix".  The wierd part was, the "sort-of" was that it worked in eclipse but not when I ran it from the command line using Gradle.

After much searching and head banging I found that it was a classpath issue.  The atmosphere-compat-jetty artifact was conflicting with the jetty implementation and depending on the order jars got loaded is whether it would work. Further searching indicated that I should exclude the compat-jetty artifact since I am embedding jetty and leave the other compat-* artifacts alone.
The exception:
java.lang.UnsupportedOperationException: Please remove the atmosphere-compat-jetty from your classpath
Should have pointed me to this, but I wasn't sure why it was included and why I needed to remove it.  After realizing the artifact existed as a placeholder until the app was loaded into a servlet container and only conflicted because I was embedding the container it was a simple matter to correct.

In Gradle this is simply:

configurations {
   all*.exclude group: 'org.atmosphere', module: 'atmosphere-compat-jetty'
}

Another useful tool in my experimentation is the following page that lets me connect to an arbitrary websocket:
http://websocket.org/echo.html

If you have any questions or comments drop me a line or post in the comments.

Linux Framebuffer Drivers

I am working on building a framebuffer driver for my own custom VGA hardware module. It only runs at 640x480 and has a dedicated video buffer in the FPGA.  Linux is being run on Microblaze, I am using petalinux because that is what I have to use (This is an academic project).

I ran into a problem with it saying that the register_framebuffer and unregister_framebuffer functions were undefined. The kernel also refused to load the module for obvious reasons.

After chasing my tail for hours trying to link against fbmem.o I realized that it wasn't even generating the object file. I also concluded that it probably needed to enable it in the kernel. So one last make menuconfig and enabled framebuffer support. "Kernel/Library/Defaults Selection>Customize Kernel Settings" then "Device Drivers>Graphics Support>Support for frame buffer devices"

http://xkcd.com/979/

Tip of the Day: Backup single file in folder with CrashPlan

I played with Bitcoin a while ago and while I have very little in my wallet I still think it would be a shame for it to be gone forever. So I wan't to keep it backed up while not backing up the constantly changing gigs of other files (I don't understand why they are in roaming...).

So here is a simple regex to backup a single file:

.*/AppData/Roaming/Bitcoin/(?!wallet.dat).*

Does not work:
.*/AppData/Roaming/Bitcoin/(?!wallet.dat)

It uses negative lookahead to not match the wallet.dat, it doesn't consume any characters so the .* consumes anything left over. It is worth noting that this also matches any files starting with wallet.dat so you could also use this to exclude all files except ones prefixed with something or exclude everything except a specific folder.

Progress: Better late than never.

So I've been working around the clock trying to get this project done by the end of the term.  Still a ways to go but it may be possible to achieve "Functional" by the end of this weekend. You never know, if I am a god I may pull off feature complete as well. There is no way however a single item on the wishlist is going to make it.

So here are the highlights of what I've been doing since I last posted:

I completed the emulator for the coach side of the thermostat. It has led's for heat, ac, high and low fan in the two zones as well as a button to indicate the zone is shedding. There is a nice metal power connector on top providing the important 12V to run the communication line which is connected over BNC. The choice to use BNC has made it look so good. Maybe too good because my roommate insists I take it as carry on at the closest airport, I disagree.

I completed the daughter board for the DE-0. This has all the circuitry for the temperature sensors, real-time clock(in red) and the communication line to the emulator.  The capacitor and neighboring diode provide 12V to this board to run the voltage comparators, the DE-0 only supplies 5V & 3.3V.  The empty space up top is for the analog to digital converter for temperature.  I purchased the wrong one MAX110 and wanted a MAX111, I could make due with the one I had but a wiring mishap let the smoke out shortly after seeing the temperature on screen (go figure). I have received the new part and it is working wonderfully as seen in the picture at the top of this post.

Changes to code:

1. Abandoned the use of the SD card. While communication with the card worked great, endian issues with open source FAT implementations made it not worthwhile with time constraints. Images are now stored in the ROM.
2. Implemented libpng. We emulate files using a library and use the custom IO functionality in libpng to make it all work. A simple malloc/free setup was also added.
3. Not really changed. We are using the wrong toolchain, I tried several more times to get the newlib toolchain to work as desired in cygwin with no success. Gave up.
4. Struggled with interrupt oddity.  The A/D interrupt input wasn't latched and at random would violate setup times on the OR1200 causing undefined (The return address would get corrupted) to occur. This is now latched on negative edge.
5. Struggled with odd interrupt clearing behavior. Turns out I misread the OR1200 spec and was doing it as described in OR1000.
6. Implemented some of the touchscreen code.

Todo:

1. De-bounce touchscreen. This was Dependant on the interrupt bugs which I resolved today.
2. Implement user interface buttons
3. Make main screen of UI
4. Make set time screen of UI
5. Time Permitting: Make schedule screen of UI
6. Get it checked off
7. Do paperwork
8. Throw in closet & forget about it
9. Pull out of closet fight with SD card more.

As always code is at:

https://github.com/Deathbobomega/thermostat

It's about time!

Here is what I have been working on the last few/several days. Another scrolling gradient but this time it is fully stored in the SD RAM. Double buffering wouldn't be too hard at this point but I will avoid it if I can.  A custom DMA controller has priority on the bus and transfers the next line on horizontal sync.  A number of tiny issues kept me from calling this done, subtle timing issues caused the image to be shifted right 2 pixels. Fixing some bugs made it appear worse even though it was closer to correct.

My colleague who is also using the OR1200 has been kind enough to keep a record of issues he had with building the toolchain (additional packages needed, using git for the first time, etc). I will be working with him to revamp my earlier post. I will also later be checking in code for this and finally be able to move onto implementing the SDCard.

Picture Time

This is an old program on the board (before the inclusion of the processor) it just shows the touch position.

This is the current program. The gradient is scrolling from right to left. Also note how the screen is upside down. This is due to the way Terasic setup the headers, it will only sit flat upside down relative to the development board.

Lastly the DE0 development board. You can see the SDCard slot that I have yet to implement as well as the ribbon cable leading to the touchscreen. I also find it odd that they cut the plastic shield such that it covers two of the momentary contact switches.

OR1200, Cygwin, SDRAM, Interrupts & the Red zone

So I didn't quite reach my goal of procedural images on the touch screen. I did run into some setbacks along the way that slowed me down but all of that is now resolved and I will go into that more below. The highlights of the week have been getting the SDRAM working, Getting interrupts enabled on the processor, learning about the red zone and just moments ago compiling the toolchain on Cygwin.

SDRAM

After much frustration I was unable to get a core from OpenCores that worked with the SDRAM. I had to port a bunch of code to compile on Quartus since it's Verilog support is incomplete. But all was for nothing since it didn't work at all. In the end I ripped the SDRAM module out of the DE0 SPOC example. The issue with that is that it is for the Avalon bus and a 16 bit one at that. With some finagling I built a bridge between it and the wishbone bus that takes care of most everything. There is still a bug in reads that causes more transactions to the ram than needed but I will look into that later. I will also probably need to use a wishbone-wishbone bridge to run the ram at 100MHz since at present it is taking 9 clock ticks to do a read. I am unsure if this will be fast enough to operate the ping-pong buffer for the touch screen but only time will tell.

Interrupts

I am not certain but it would appear that interrupts 0 and 1 are non-maskable. However unmasking them in the PICMR (Programmable Interrupt Controller Mask Register) is not enough. Interrupts in general must be enabled in the Supervisor Register. It is worth noting that the supervisor register can only be modified using a Read, Modify, Write method. Thus use GREAT CARE if you ever decide you need to modify a setting in the register. You need to be careful that you do not modify the register in an interrupt or exception whilst a lesser priority interrupt or the normal flow of execution is also modifying it (Your changes will be lost). My recommendation is set it and forget it. After that, only modify the register for critical sections (disable/enable interrupts) and everything will be fine.

Red zone

It is a little known fact (I didn't know it till 4AM this morning) that the OR1200 toolchain utilizes a Red zone. This means that code is free to use 128bytes beyond the stack pointer for temporary data. Which also means that your interrupt/exception handlers need to decrement the stack pointer by 128 on entry or you will screw something up (makes for a psychedelic experience on my touch screen). This isn't really documented and only after I asked the IRC about why GCC was generating negative stack references did I find out about it.

Cygwin

I got back to trying to make Cygwin work, I had long abandoned the subversion repository in favor of the one in GIT for my compilation on my laptop. I tried this on Cygwin and the results were less than satisfying. After some digging I found out why.

So heres how to get a working toolchain in Cygwin that can compile both c and c++ for the OR1200.

[Check comments for better tips on setting up cygwin from scratch]

Clone the GIT repository (Do not build!) following the instructions at http://openrisc.net/toolchain-build.html

It is about 600MB so it will take a while.

Now we need to patch the uClibc build process.

So modify Makefile.in in the uClibc folder.

https://bugs.busybox.net/show_bug.cgi?id=3589

This bug has the lines that need to be changed.

Make sure you have the flex package in Cygwin

Now run "make". Thats it, no command line parameters.

Wait 1-3 hours.

When it is done the toolchain-out directory should have everything you need. I am still refining my makefile so there may still be kinks in that department.

Conclusion

DO NOT USE THE VMWARE IMAGE EXAMPLE

That is basically the conclusion, don't do it. I have made so many subtle changes to the example it isn't even funny. BootReset.S in my firmware directory is the modified/fixed/hacked remanence of that example. At the very least take it but I recommend salvaging as much of the helper stuff from my code as much as possible. I am still working on it so I should improve over the next month and a half.

As always you can find my latest code here:

https://github.com/Deathbobomega/thermostat

Till next time,

Thomas

OR1200, Github, Touchscreen

I worked out the kinks that I was tracking down. Turns out that while I was going to change the clock edge of my RAM after I found the kink I should have just done it anyway. I was corrupting random memory with every write, fun times.

So here is where I am placing code now: https://github.com/Deathbobomega/thermostat

I am using the TerASIC Touch Screen for my user interface. This is finally hooked to the processor in some fashion. At time of writing the ping pong buffer is only partially implemented and the off chip ram is not yet used at all. This means it just displays one line of the screen across the entire screen. It also does a small animation to make sure it is working.

I hope to have the off chip RAM and procedural images by the end of the week. Then I can move onto the SD Card and PNGs.

OR1200 & Altera

It has been a while since my last post. I came across some bugs in the Quartus II software that severely slowed down my progress. Some of that was my fault since I just couldn't let it go and spent hours trying to find a work around only to fail.

So in the end instead of having a parameterized traffic cop for the bus I ended up with coding every signal on the bus into the traffic cop. This means every time I add a new device on the bus I get to add more wires by hand which is not speedy and is error prone (Killed a few hours because I had a typo when I made it bigger). I suppose it is better that using Xilinx and not having SystemVerilog at all.

This is even more motivation to make my own HDL compiler which I am pushing forward as my Software Engineering Senior Project next year. I need to buckle down and write the idea proposal that is due next week.

The other bug in Quartus that is annoying me is array literal syntax for interfaces '{ interface_a, interface_b } is broken. Nothing gets wired, sorta. From what I could tell a few signals were half hooked up when I inspected using SignalTap. 2 wires got a value but the other end of those wires didn't. It was weird. I have sent these off to Altera and they acknowledged them after a bit of a hissy fit about me not using a supported linux distro (Even though, they could reproduce the problem themselves).

I've been working on the boot loader the last few days but it still has a few kinks in it I want to fix before pushing any changes. Not totally sure whats going on but it may be a problem with the traffic cop (though I hope not).

OR1200 processing instructions

Well, after a tiny amount of sweat and tears I got the OR1200 hooked to a ROM for instructions and was able to execute them. The instructions I am currently running are from the example in the VMWare image. Before I actually sit down and write any C code myself I am investigating linker scripts and how to do what I want.

Unfortunately Quartus does not support parameterized filenames in the Verilog $readmemh command. This means that the filename must be hard coded and the module is single purpose. Bummer. I submitted a feature request to Altera but my hopes aren't high.

Here is the code from what I did:

http://pastebin.com/27b7WXFq

http://pastebin.com/LbUQjDRF

http://pastebin.com/mW3mdfMH

http://pastebin.com/SbTTFJUB

It is worth noting that I did not hook anything to the data Wishbone Bus. Instead I asserted retry so it will infinity loop trying to perform a bus transaction. Eventually I will insert a traffic cop and combine the buses.

Quartus Programmer [Linux]

In other news, my laptop is running fedora instead of Windows 7 like my desktop. I haven't used my laptop to program my board before so I gave it a go. Turns out its a pain. There is a permission problem, so you need to create a udev rule. After a bunch of research and partial solutions to the dreaded "Unable to lock chain (Insufficient port permissions)" problem I crafted the following udev rule which seems to work nicely. This issue is udev changes the permissions in /dev but quartus uses /proc so we need an extra command to update that as well.

BUS=="usb", SYSFS{idVendor}=="09fb", SYSFS{idProduct}=="6001", SYMLINK+="usbblaster", MODE="0666", RUN+="/bin/chmod 0666 /proc/bus/usb/$env{BUSNUM}/$env{DEVNUM}"

OR1200 Continued

Working on making progress on the processor. Thus far I have http://pastebin.com/qSLuvy3X

It uses the RAM_wb IP for the instruction RAM (With a few bug fixes). This doesn't really provide a way to load a boot rom as far as I can tell but I will worry about that later when *something* works. I've also been fiddling with the VMWare image which involved replacing hard coded IP addresses with opencores.org in three scripts. I'll see later if I can at least update two of them myself. The first is unfortunately embedded in the VMWare image, I may still be able to update it and see about making it also work with VirtualBox.

I am surprised that anyone has ever used this stuff and gotten it to work. No wonder companies are willing to fork out so much money for a solution.

OR1200: My first impressions of OpenRISC

Alright, so I'll prefix this with saying I am working on my Senior Project for my Computer Systems Engineering degree.

My project includes a touch screen interface with a small amount of graphics. Implementing this in pure hardware would be a state machine nightmare so I have chosen to include a soft-core processor in my design. I am using the DE0 development board which includes an Altera Cyclone III FPGA. I'm sure there are fine Xilinx boards out there but I am tired of the ISE not supporting SystemVerilog and having to generate an EDIF in Precision only to watch the ISE puke its guts on the floor during programming. This also means I don't have to pull a Precision licence out of my arse. For similar reasons, I wanted to avoid the NIOS processor core so I wouldn't have to deal with licencing. Sure, Altera is really nice to students and would undoubtedly provide me with a 1 year licence if I asked nicely but that is beside the point.

So I created an account on open cores and acquired the processor code via svn at http://opencores.org/openrisc,or1200

I was lazy so I threw all of the files into Quartus and recompiled. Nothing was totally broken so I took a moment to put my feet up and contemplate which file had the module I should instantiate. Overwelmed, I returned to the site and began reading the tutorial at http://emsys.denayer.wenk.be/?project=empro&page=cases&id=6

I found the top module and instantiated that without connecting anything. Compiling revealed that it appears to compile despite having it all optimized out. I still don't know what to do about the wishbone buses and decided to download the vmware image (which I have yet to try). The link on the site is broken but the file can be found at http://opencores.org/ocsvn/or1k/web_uploads/vmware_image/

I'll post here as I figure out more.