Sunday, January 29, 2017

Raspberry Pi, Multicast, UDP, & WIFI

I've been working on a project recently to synchronize multiple raspberry pi's playing videos. The goal was to be within 50ms (about 1 frame) of each other. I'd started testing with a Pi-3 and a Pi-1B but quickly realized the Pi-1B was going to be painful to diagnose issues with the synchronization vs just being slow. The Pi-1B took a LOT longer to start playing a video and had more latency in seeking/changing speed. So I switched to testing with two Pi-3's and to my surprise, it was worse. I banged my head for weeks trying to figure out why different configurations behaved radically different. Data I collected with a packet sniffer was contradictory. In some configurations, specifically any configuration where a Pi-3 was the slave in the synchronization scheme, the latency from the master Pi had a jitter of 300ms. I would receive 10-20 synchronization packets at once then there would be silence for 300ms. With the Pi-3's I had been using wireless because for convenience that would be the easiest way to use them in production. I was guessing going to a wired connection would correct the issue but it bugged me that a wireless link with <2ms of latency with the ping command regardless of packet size would have such a wild jitter with UDP packets. It took a long time for the significance of the Pi-1B being wired and the jitter only appearing when the Pi-3 was a slave. It didn't help that when I used Wireshark on my desktop the packets came spaced give or take 2ms, in contrast my laptop would show the jitter. I was doing each test on a different day and then last night it finally hit me, Multicast packets from a device to my rt-ac66u over wireless or wired have little latency when viewed on a wired connection like my desktop. In contrast, those packets were being buffered and delivered to all wireless devices only 3-4 times per second. ICMP packets were being shipped same-day, but UDP multicast/broadcast were being snail-mailed in batches. GRR. I never did find a setting on the router to alter this behavior and have simply conceded to using wired connections for the Pi's. If anyone has thoughts on this phenomenon, I would be glad to hear them in the comments.

Monday, July 21, 2014

Google-Drive, Fuse, and Docker

Today I was fiddling with an idea I had of using Google-Drive to store configuration information for docker images. It is much too slow to do anything that would hit it on a regular basis, but for configuration I think it could work quite nicely.

On that note I went out to explore the idea of a Google-Drive docker image. The idea being you would use the volumes-from capability to access the information from the container connected to google drive which would also likely need extra privileges.

On that I was correct, the internet recommends time and time again that to get Fuse to work you need

--privileged=true

I was curious if there was any way around that and found that indeed it is possible to only allow access to fuse by instead using

-v /dev/fuse:/dev/fuse

That maps the device into the container and surprisingly works.

For those interested in my dockerfile:

FROM base/archlinux

RUN pacman -S --noconfirm python2-pip fuse
RUN pip2 install ez_setup
RUN pip2 install --allow-unverified antlr-python-runtime --allow-external antlr-python-runtime google-api-python-client gdrivefs


You will need to get credentials as described in https://github.com/dsoprea/GDriveFS

Tuesday, September 3, 2013

Wordpress

I thought it was about time I start using Wordpress for my website. Actually, for quite a while I have wanted to switch to using a CMS instead of using some custom integration with phpBB. I had dabbled in Drupal and Joomla and a few others and always got frustrated. I installed Wordpress a long time ago and didn't make much progress.

However I had decided to give it another go since Dreamhost added one click install. The admin interface appears to have matured and after some googling to get a decent blank slate theme I have started work on porting my current sites theme to Wordpress. The current site is full custom and I have been wanting to move away from the dark theme for a while, however I am in love with the general layout and wanted to preserve it. I have made decent progress on core layout and modernizing certain elements of the design, including dropping support for IE <8.

One annoyance though was the new Admin Bar (or Toolbar). My theme utilizes a height:100% style on html and body. The admin bar adds a margin to the top of html which made my theme scroll funny. This only matters if you are logged in (aka, just me) but it pissed me off so I wanted to fix it. After scouring the internet I pieced together the steps to unhook their addition of the margin and add my own hook to adjust the padding instead. Just add this to functions.php in your theme:

function pad_admin_bar() {
  echo '
    <style type="text/css">
      html {
        padding-top: 28px !important;
        box-sizing: border-box;
      }
    </style>';
}
function my_admin_bar_init() {
  remove_action('wp_head', '_admin_bar_bump_cb');
  add_action( 'wp_head', 'pad_admin_bar' );
}
add_action('admin_bar_init', 'my_admin_bar_init');

The box-sizing trick makes my height style still work as expected. Now I can be happy and move on to the parts of the theme normal people see.

Friday, February 3, 2012

Akka Untyped Actors & Google Guice

I started using Akka Actors in my Scala project and wanted to make them play nice with Google Guice.

Akka Actors are STRICTLY (I don't think you understand how strict) enforced to be instantiated in a certain way.  So I went on a trek to do this.

First off, Akka Actors are wrapped in an ActorRef. This ref has no reference to the underlying Actor type. An ActorRef is an ActorRef is an ActorRef. So to differentiate them you must use annotations. Not wanting to make a new annotation for each I went with the provided @Named annotation. This leads to a not so terrible injection of:

class WeaveCompiler @Inject() ( @Named("WeaveActor") val weaveActor:ActorRef ) {

That's all fine and dandy, and satisfies the first half of the binding:

bind(classOf[ActorRef].annotatedWith( Names.named("WeaveActor") )

But now we need our ActorRef. We can't let Guice do it for us since we have an Actor and it needs an ActorRef. Time for a provider. I didn't want to have to make a provider for every single Actor so I made it generic. Java type erasure quickly bites us in the ass but there are ways around that there are over my head. In short, the following works:

.toProvider( new TypeLiteral[ActorProvider[WeaveActor]]() {} )

So TypeLiteral is provided by Guice and takes care of that type erasure headache. The ActorProvider is my own special glue that I will provide below and WeaveActor is my Actor that extends akka.actor.Actor

So here is my glue:

import com.google.inject._
import akka.actor._

class ActorProvider[T <: Actor] @Inject() ( val m:TypeLiteral[T], val injector:Injector ) extends Provider[ActorRef] {
  def get = {
    Actor.actorOf( injector.getInstance( Key.get( m ) ) ).start
  }
}

Because of how akka actors work the Actor object MUST be created inside a call to actorOf. Fail to do that and you get a horrible exception.  Thus the provider gets the TypeLiteral for the actor injected with the Injector from Guice. Then uses those to get an instance of the Actor from the Injector inside the call to actorOf. Then it starts the actor and returns the ActorRef.

This solution seems simple, but I spent the better part of 4 hours to figure it out.  There is probably still something missing (I haven't checked what happens if you inject the same actor in multiple places) but that can be an exercise for the reader. If I find any problems I'll make another post about it.

Hope this helps!

Wednesday, February 1, 2012

Tip: Guice Multibinder & Scala Set


I am using Guice Multibinder with Scala and this took me way too long to think up. To make things nice I want Scala Sets not Java Sets. So the following is a nice way to do it, make a constructor with Java types and call the one with Scala types.

import scala.collection.mutable.{Set => MutableSet}
import scala.collection.JavaConversions._

class MyClass( val t:MutableSet[T] ) {
@Inject() def this( t:java.util.Set[T] ) = this( asScalaSet(t) )

//...

}

Thursday, January 19, 2012

Parboiled, Case Classes and Headaches

I had the lovely task today of figuring out how to not use case classes in Scala for my AST.  My AST demanded as structure that resulted in conflicts of overriding constructor parameters when case classes were used. This meant I needed to build the class myself, and it took all day to get it right. My first attempt was the following as I found a blog post stating that Parboiled only depends on the apply method:

object ImportViral {
  def apply( packagespec:PS ):ImportViral =
    new ImportViral( packagespec )
}
class ImportViral( packagespec:PS ) extends ImportStatement( packagespec ) {}


And my respective rule in the parser:

def ZImportViral = rule { "importviral" ~ WhiteSpace ~ ZPS ~~> ast.ImportViral }

But this resulted in a rather ugly error:

error: overloaded method value ~~> with alternatives:

[X, Y, Z, R](f: (X, Y, Z, ast.PS) => R)org.parboiled.scala.rules.ReductionRule3[X,Y,Z,R] and

[Y, Z, R](f: (Y, Z, ast.PS) => R)org.parboiled.scala.rules.ReductionRule2[Y,Z,R] and

[Z, R](f: (Z, ast.PS) => R)org.parboiled.scala.rules.ReductionRule1[Z,R] and

[R](f: ast.PS => R)org.parboiled.scala.rules.Rule1[R]

cannot be applied to (net.codingwell.weave.languages.silk.ast.ImportViral.type)

I figured out pretty quick that the issue was f: ast.PS => R meaning it wanted some sort of conversion from my PackageSpecification to "R" (ImportViral). So obviously I did something wrong. I fiddled for half the day before trying hardcoding the rule type.

def ZImportViral:Rule1[ast.ImportViral]

This gave me a new only minorly more helpful error.

found   : net.codingwell.weave.languages.silk.ast.ImportViral.type (with underlying type object net.codingwell.weave.languages.silk.ast.ImportViral)

required: net.codingwell.weave.languages.silk.ast.PackageSpecification => net.codingwell.weave.languages.silk.ast.ImportViral

So again, my object isn't a conversion. After another several hour sprint trying to figure out what a case class is equivalent to in code. I found a decompiler http://java.decompiler.free.fr/?q=preview
Plugged in a case class from my AST and looked for differences. Took me a little bit to notice but plain as day, the object was inheriting from Function1. The magical glue to make a function object in scala. Now I am no Scala expert so this also taught me that objects can inherit from different things than their companion class. Who Knew?

Final working code:

object ImportViral extends Function1[PS,ImportViral] {
   def apply( packagespec:PS ):ImportViral =
     new ImportViral( packagespec )
}
class ImportViral( packagespec:PS ) extends ImportStatement( packagespec ) {}


This code can be found on github: Github - Deathbobomega/Weave - ast.scala

Wednesday, December 28, 2011

Embedded Jetty, Atmosphere & Jersey

Backstory
So, my final major project for my bachelors degree's is software in nature, though it still has it's roots in hardware.  My project is to build a compiler / translator for my own custom HDL titled Silk. The compiler (named Weave) will output a Verilog file for import into a standard synthesis tool. Written in a mix of Java & Scala it is hoped to eventually integrate with Eclipse.

14.7 PSI
Atmosphere is an abstraction layer for bidirectional communication in web applications. You are probably thinking, Weave is a compiler not a webapp... You would be correct.  However, with more and more moving to the web it would be extremely convenient to have a websocket api for the compiler. At this point I haven't exactly figured out what atmosphere gives me over using the Jetty API directly.  The documentation for atmosphere is non-existent at best and outdated / wrong at worst.  It seems to have a lot of hype / people swearing by it so I will give it a shot.
To get started however I spent the last few days getting it compiling and running.  At this point I now have an embedded jetty server and atmosphere accepting websocket connections. I am unsure as to what the protocol is and will begin looking into that next as well as setting up connecting to the server on the other side.
So without further ado...

The Code
So first off here is the code that I had when I wrote this blog post:
https://github.com/Deathbobomega/Weave/tree/9a77f3d2b688a906f8ecc2c5dcd68452fb272018
For those who are also interested in continuous integration here is the build server:
http://ci.codingwell.net/job/Weave/12/

I do believe I would have never figured this out (or would have required several back and forth posts on the mailing list) if it wasn't for Charles Brown. He posted some example code to github that I found extremely helpful.
https://github.com/charlesbrown/Jersey-Atmosphere-with-Embedded-Jetty

I however ended up chasing my tail for hours/days trying to figure out why I had the same problem he did:
http://markmail.org/message/6wxzztdwbgfgxmp3

But his solution only sort-of worked for me. This also made me wonder why the github code (which was newer) did not include this "fix".  The wierd part was, the "sort-of" was that it worked in eclipse but not when I ran it from the command line using Gradle.

After much searching and head banging I found that it was a classpath issue.  The atmosphere-compat-jetty artifact was conflicting with the jetty implementation and depending on the order jars got loaded is whether it would work. Further searching indicated that I should exclude the compat-jetty artifact since I am embedding jetty and leave the other compat-* artifacts alone.
The exception:
java.lang.UnsupportedOperationException: Please remove the atmosphere-compat-jetty from your classpath
Should have pointed me to this, but I wasn't sure why it was included and why I needed to remove it.  After realizing the artifact existed as a placeholder until the app was loaded into a servlet container and only conflicted because I was embedding the container it was a simple matter to correct.

In Gradle this is simply:

configurations {
   all*.exclude group: 'org.atmosphere', module: 'atmosphere-compat-jetty'
}

Another useful tool in my experimentation is the following page that lets me connect to an arbitrary websocket:
http://websocket.org/echo.html

If you have any questions or comments drop me a line or post in the comments.