Yeah, I’ve been singing the praises of static typing for a few weeks now, but it’s time for me to admit some of the faults I’ve run into even in a well thought out static typing system like in Scala. While I’ve been wholly converted to the static typing camp, these problems I’ve found bother me. The one I’m highlighting here emerged while working on SNMP4S.
One of the key features of SNMP4S is type-safety derived from compiling the MIBs which define the SNMP interface. Think of it as a WSDL for an SNMP interface. Each object that you can manipulate via SNMP has a MAX-ACCESS property (Read-Only, Read-Write, Not-Accessible, etc) and a type (typically OCTET-STRING, INTEGER32, and enumerations), both of which I have encoded as SNMP4S types. Then with each object, you can get or set the current state. Since these operations are I/O bound on the network, you want to batch all of the gets or sets into a single packet. Hence, you need
set() to accept a variable number of arguments. Initially you may think that you can implement this by having get/set accept a sequence of objects such as this:
def get[A <: Readable, T]
Either[SnmpError,Seq[Either[SnmpError,T]]] = ...
Unfortunately, this won’t work for all cases. It restricts all of the arguments to have the same access
A and type
T. You won’t be able to mix a read-only object of type
String with a read-write object of type
Int. To do that, you must have two explicit arguments with the type annotations for
def get[A1 <: Readable, T1, A2 <: Readable, T2]
(obj1:DataObject[A1, T1], obj2:DataObject[A2, T2])
(implicit m1:Manifest[T1], m2:Manifest[T2]):
Either[SnmpError,(Either[SnmpError,T1], Either[SnmpError,T2])] = ...
That does work for the case of getting two objects, but what about three? Four? N? To support that, we must have a function for each number of arguments we want to support. Hence, it’s not possible to support an arbitrary number of objects. We have to accept an upper bound. Picking the upper bound is easy, because it’s 22. So I have to write 22 functions for each operation. Wow, that sucks. Limited functionality AND copy/pasting.
The good news is there is sbt-boilerplate to code this up for me. You just define the base case and it generates all 22 cases from it. This is what my
get() declaration looks like in the boilerplate’s template language:
[#def get[[#A1 <: Readable, T1#]]
Either[SnmpError,([#Either[SnmpError,T1]#])] = ... #]
And it works beautifully! Each snippet surrounded by
[##] gets repeated 22 times, generating my functions in a scala source file. Write some tests… Everything is working well. Generate the documentation and, uh-oh… 22 functions named
get which all do basically the same thing. The scaladoc page is bloated with nasty entries like this:
get[A1 <: Readable, T1, A2 <: Readable, T2, A3 <: Readable, T3, A4 <: Readable, T4](objs: (DataObject[A1, T1], DataObject[A2, T2], DataObject[A3, T3], DataObject[A4, T4]))(implicit m1: Manifest[T1], m2: Manifest[T2], m3: Manifest[T3], m4: Manifest[T4]): Either[SnmpError, (Either[SnmpError, T1], Either[SnmpError, T2], Either[SnmpError, T3], Either[SnmpError, T4])]
That’s just the case where n = 4! You can imagine how ugly the 22-arg one is. Add to that the fact that there are 22 of these. Good luck finding that now-obscured function
walk(). Hopefully someone will tell me how to exclude these from scaladoc generation. Otherwise, this critical documentation will be very painful for potential users of my DSL. For now, I’ve decided to not worry about the documentation clutter. Hopefully my published examples will keep folks on track.
I used boilerplate to produce my
get() functions as described above, as well as my
set() methods. Then I decided it could be handy if
get() could also accept a tuple of objects. (I won’t get into why I thought that would be handy.) Well then I hit another problem. This time, it’s a bit more problematic than unusable documentation. Check out how long it took to compile and run my tests:
[info] Passed: : Total 21, Failed 0, Errors 0, Passed 21, Skipped 0
[success] Total time: 904 s, completed Jun 6, 2013 1:41:40 PM
Yes, you see that correctly. 904 seconds, or 15 minutes and 4 seconds.
Prior to adding the
get(TupleN), the time needed to compile my test suite was getting slow, but not THAT slow. After removing the tuples:
[info] Passed: : Total 20, Failed 0, Errors 0, Passed 20, Skipped 0
[success] Total time: 44 s, completed Jun 6, 2013 2:19:37 PM
Not only does the compiler take an unacceptable amount of time to run, my eclipse IDE hangs up for long periods of time while trying to save a file. I greatly appreciate the extra work that the Scala compiler does for me, say compared to javac, but this is a sure way to have an unusable library. Needless to say, I threw that tuple version of
Cluttered, useless documentation. Slow compilation. Frozen IDE. Stupid static type checking.
Now, it’s possible that shapeless is the solution. I was able to get some toy code working with this library. Unfortunately, I found myself wrapped around the axle of Scala higher kinded types. After some discussion, I had decided for now I’d just use sbt-boilerplate. Too bad that led me into these problems. I’ll probably revert back to shapeless eventually in hopes of making all of this better. For now, I want to hammer out some real value in this DSL before I make it pretty.
So there ya go. The good and the bad. Prose and conz… Yes Mom, I’ll get around to writing about lambdas before I drop this static typing discussion…