prose :: and :: conz


Java properties made elegant in Scala

If you’re no rookie to the JVM world, you are certainly familiar with the necessity and pitfalls of Java’s System properties. They’re ubiquitous in the Java world, yet not always trivial to deal with. It also seems no one shares the same opinion on how they should be done. In this post I’m going to demonstrate what I did with them in Scala earlier this week, and my opinion is sure to clash with many of the five or six readers of my blog. While this is a bit of an intro to some features to Scala, even the seasoned Scala developer may appreciate the approach I arrive at in the conclusion. For those interested, I have started an open source project featuring this approach named omniprop on github.

Why properties

First we should have an understanding of why we would need properties. In particular I’m writing about their use in applications as opposed to libraries or APIs. They’re highly useful for manipulating parameters of your application without requiring a new build of the application artifact(s). You may need to do this to configure your application for different environments (development, test, staging, deployment, etc). A common example of this usage is for the application’s database URL. The application code itself needn’t care where the database lives, or even the flavor of the database (in the case of SQL). It only needs to be able to discover the location of the database at run time, whether it be an in-memory H2 database for development, a staging database with a handful of rows per table, or the production database with all of your end users’ data.

Another case I’m interested in is any tuning parameters. For my current project at work, I have timeouts, retry counts, and actor pool sizes which all could have an impact on the performance of my application. During development, I can only guess at the optimal settings for these parameters. Once we’re in staging and eventually production, we will learn much more about these settings and what may be more optimal for the load we eventually incur. These are certainly not the sort of things we want to require an entire rebuild of the application to tweak. I need to be able to change them externally to the build artifacts.

Problems created

As it always seems to be the case, when you solve one problem with software you create another problem. Java properties are no exception. I’d argue it produces at least four problems. It’s just this global map of string keys and string values. Anybody can add key/value pairs, and anyone can expect certain key/value pairs. Names clash (1). Even when named well, there’s no sure way to find them all in the code outside of find . -type f | xargs grep prop with a dash of hope that no one programmatically built the string (2). After clearing any hurdles there, you have the challenge of converting the string values from this otherwise untyped object into the statically-typed JVM world (3). Then there is the concern of whether or not the property was defined in the first place (4).

Some solutions in Java

Let’s first look at how I would solve these snags in my past Java days. For starters, I like to name my application’s properties with the fully-qualified package name of where I need to use it. If I have a class named com.joescii.MyJavaClass, then I want that class’s properties to be prefixed with that full class name such as com.joescii.MyJavaClass.prop. This solves the first problem by drastically reducing the probability you happen to bring in a library or whatever that wants to use the same name. It also solves the second problem by identifying precisely where in your code you utilize the property.

The next two problems are typically solved by writing your own PropertyUtil class, or you just use someone else’s like Typesafe’s config. You have your getInt(), getBoolean(), etc. methods that will do the conversion for you. If it did not parse correctly or otherwise wasn’t defined, then you either except out or have some way to provide defaults.

An implementation of these ideas may look something like the following code. I’m omitting the PropertyUtil implementation since it is rather irrelevant for this discussion.

package com.joescii;

public class MyJavaClass {
    public static final int prop = 
        PropertyUtil.getInt("com.joescii.MyJavaClass.prop");
}

While this works for my four earlier complaints, again we introduce more problems. The most notable is that the name of my property is repeated and utterly not DRY. I want it to reflect precisely where the use of the property can be found in the source, but I had to copy/paste the name by concatenating the package, class, and field names. If I later refactor, I must lean on good tooling like IntelliJ to help me get it right. It would be much preferable if it figured out its own name perhaps using reflection. We can partly get there, as you can see in the following snippet.

public class MyJavaClass {
    public static final int prop = 
        PropertyUtil.getInt(MyJavaClass.class.getName()+".prop");
}

That’s a little better. It has some boiler plate in there to grab the class name. The name of the field is still copy/pasted over. There’s not a good way around this because it is a field. There is no introspection we can do on this field to figure out that its name should be prop.

With only my Java glasses on, this looks OK. It’s open to refactoring mistakes. The property doesn’t feel strongly tied to its type either. Developers who get around more than just Java may likely remark that this is a very imperative way to express the property. The code does its job, but it’s nothing to be excited about.

Making it elegant in Scala

Scala offers an incredible amount of language malleability which lends itself well to writing domain-specific languages (DSLs). This can be used in your everyday programming such as this property problem. What we want is to be able to declaratively create a property which is typed correctly and nothing is copy/pasted to form the name. I don’t even want to repeat MyJavaClass as we did above. Guess what… All of this can certainly be done.

First off, we won’t make our property simply a val on our class. We will make a companion object and make the property an object within the companion. If you aren’t familiar with Scala objects, just for this discussion regard it as a singleton. While that is only one use case, it will suffice for now. A companion object in particular can be thought of as the space for the static fields and methods of a class. Let’s take a look.

object MyScalaClass {
  object prop // ...
}
class MyScalaClass {
  // Instance stuff...
}

What’s interesting about my use of object for the prop is that it is an object, and hence has a class. We can perform reflection on this object, and get a fully-qualified name including the package, class (or companion object to be more precise), and field name. The following spec demonstrates this:

class PropertySpecs extends FlatSpec 
  with ShouldMatchers {

  "The prop" should "have a full class name" in {
    val name = MyScalaClass.prop.getClass.getName 
    name should equal ("com.joescii.MyScalaClass$prop$")
  }
}

Now the uninitiated may be puzzled by the dollar signs in the classname. While hijacking of the JVM is troublesome for the occasional developer, this is one approach to finagling an alternative language into the Java runtime. But don’t let yourself get distracted by that. Notice that the class name has everything we wanted in it. The package, class, and field! It just needs a little massage:

"The prop" should "have a full class name" in {
  val name = MyScalaClass.prop.getClass.getName.
    replaceAll("""\Q$\E""", ".").
    init
  name should equal ("com.joescii.MyScalaClass.prop")
}

The replaceAll converts the $’s into periods, and the init returns all but the last character, effectively dropping the trailing $.

Now that we have our ability to name it, we need to put that code into a class that our property objects can extend. Let’s start with a properties package object and put our code in a trait.

package object properties {
  trait Property {
    val key = this.getClass.getName.
      replaceAll("""\Q$\E""", ".").
      init
  }
}

…and update our prop object

import properties._

object MyScalaClass {
  object prop extends Property 
}

And now we can just refer to key to get that nice name. Although this only defines the property key, it’s already a substantial improvement over our Java counterpart from earlier. What’s cool is we can move right along and include the typed value as part of this property. After all, the key is just a means to the end value. Let’s now declare a get function on our trait.

trait Property[T] {
  val key = this.getClass.getName.
    replaceAll("""\Q$\E""", ".").
    init
  
  def get:T 
}

Our trait now has a type parameter (or a “generic” in Java 5.0+ terms) so regardless of the property type, we can utilize this code. Obviously, we shouldn’t define the integer parsing logic on our particular prop object. Instead we should now define an IntProperty to house this code for sharing this with other integer-typed properties.

trait IntProperty extends Property[Int] {
  def get = PropertyUtil.getInt(key)
}

Again, I use my PropertyUtil to hide uninteresting details. Thanks to type inference, get is properly typed as returning an Int. We can update our specification to set the property before running the suite and verify that the value is correctly retrieved.

class PropertySpecs extends FlatSpec 
  with ShouldMatchers 
  with BeforeAndAfterAll {

  override def beforeAll = 
    System.setProperty(MyScalaClass.prop.key, "42")

  "The prop's value" should "be Int value 42" in {
    MyScalaClass.prop.get should equal (42)
  }
}

Who wants to explicitly reference the get value? I know I don’t. I want to think of my prop as simply the Int value itself, like this:

class MyScalaClass {
  import MyScalaClass._
  val calculate = prop + 1
}

(If you’re take aback by the import, it is how you bring in all of the values available in the companion object). Here I’m just treating the prop as an Int. We can actually do this. This is where an implicit conversion comes into play. Let’s add one to our properties package.

package object properties {
  // ...

  import scala.language.implicitConversions
  implicit def IntProperty2Int(p:IntProperty):Int = p.get
}

Thanks to our import of properties._, we already have the new implicit conversion in scope. Now we’re in business and we can write a specification which verifies this behavior.

"MyScalaClass.calculate" should "be 43" in {
  val inst = new MyScalaClass
  inst.calculate should equal (43)
}

Finally, depending on whether or not you can change your system properties at runtime, and how often your code will invoke these property lookups, you may opt to define the get member of your sub-traits of Property as a lazy val instead of a def.

Taking it to the next level

Let me start by saying the next part DOES NOT WORK. But it would be awesome. I would love it if I could pass a constructor argument to a trait. Then we could make a Default trait and mix it in as needed.

object MyScalaClass {
  object prop extends IntProperty with Default(10)
}

The trait definition would look something along these lines (again, not legal Scala)

trait Default[T](val default:T) extends Property[T] {
  lazy val get = try {
    super.get
  } catch {
    case e:Exception => default
  }
}

That would be awesome. I don’t know the reasons why a trait cannot have a constructor argument. Most of the discussions regarding that ability I’ve found date to the ancient era of Scala circa 2008-2009, so I’m not sure how much of that is still applicable. Alas, we must resort to converting our leaf Property traits into abstract classes with overloaded constructors to get this functionality.

Conclusion

Elegance isn’t everything but damn… I love this approach of declaratively defining ordinary properties in my companion objects. I feel good about not clashing with other property names. I know exactly where to find the usage of my properties. It’s completely DRY, without need to repeat any names, types, or boilerplate reflection code. It just looks right. With this in place, I can smoothly externalize any properties of my application which I imagine could be worth tweaking in different environments or in response to conditions in those environments.

The usage of this approach plays out well. Let’s say you have a pool of some resource such as actors.

package com.joescii

object MyActor {
  val poolSize = 10
  lazy val pool = ActorPool(props(), poolSize)
}
class MyActor extends Actor {
  def receive:Receive = // ...
}

You may decide that you’d like to configure that poolSize externally to the program. The refactor is very straight-forward. You only change the poolSize declaration and add the necessary import(s).

package com.joescii

import properties._

object MyActor {
  object poolSize extends IntProperty
  lazy val pool = ActorPool(props(), poolSize)
}
class MyActor extends Actor {
  def receive:Receive = // ...
}

Then you just line the property name up with exactly where the property is read in the code.

com.joescii.MyActor.poolSize=10

I feel like I should exclaim “MWAH! BEAUTIFUL!” like some kind of code chef:)


Olde Comments
  1. snemarch says:

    I’m not a big fan of reflecting the classname – while prefixing properties with TLD.MYENTITIY.MYMODULE identifiers is a good idea, binding these *directly* to a JVM fully-classified classname IMHO isn’t – it could mean a lot of headaches down the line if you need to refactor things, and if you’ve got properties that are only relevant insides implementation classes (perhaps a bit dirty, but not unlikely) you could get some really ugly property names.

    You could still handle things declaratively, though, at the expense of a bit more code – decorate your property-holding class with an attribute stating the property ‘namespace’.

    • barnesjd says:

      Well, keep in mind that you can create these objects anywhere, hence it needn’t always reside within an implementation class necessarily. One should still use good judgement as to where these property objects are defined. I really like this approach because if you choose NOT to name the property exactly the same as its location in the source, you have to name it twice. I also like the effects of refactoring. This approach keeps it cleaned up, rather than moving stuff around and the name getting stale with the current location of the property’s usage. FWIW, the omniprop project allows (er… WILL allow) the developers to do properties the old-fashioned way but with the benefit of type conversion and stacking property APIs.

  2. Adam Holden says:

    So I tried to follow this, but it got pretty hardcore pretty fast.
    Now, maybe I’m misunderstanding what you are wanting to do, but why not use a config file? Like this: http://commons.apache.org/proper/commons-configuration/

    Now I understand your concerns about keeping it DRY, but considering that you should be very careful about what you expose to the user, it’s probably worth it to keep up with those kind of things. Lastly, I think the example that the Apache Commons site brings up is interesting because it uses the same property in 2 places.

    • barnesjd says:

      I certainly could utilize the Apache Commons Configuration project you mentioned. You can replace the PropertyUtil with the config object shown on their home page. The problem I’m trying to solve is the naming of the property itself. On the Configuration project home page, this is what "number" is. At some point, you have to name these properties where you define them (in a property file or whatever) and where you read them in the code. In this blog post, I proposed taking advantage of Scala’s object construct and reflection to construct the name in the code.

      As for using a file, that isn’t always convenient. Java properties can indeed be configured in a property file, and that is often the way to go. However, they can also be passed to the JVM when launching the application (via -D command-line switch). In my current project we are utilizing Amazon’s Elasticbeanstalk service to run our web application, and you can specify property values as part of the application instance’s configuration. This use case is important because some properties are deployment-specific, such as the base URL of the application. Files have to either be baked into the application (hence cannot be tweaked after build time) or thrown somewhere in the Amazon S3 storage service.

      I agree 100% you should be careful about what you expose to the user. I don’t believe properties are ever an appropriate mechanism to allow an end user to configure an application. They are great for developers of that application to tweak because they contain some intimate details of the application and require care.

      As for using the same property twice, that is doable with my approach. You simply reference the property object in how every many different places in the code you like. My approach has the advantage that you only cross the boundary between the code and the properties in one location, and all references to that location are easily tracked.

  3. […] my recent post on Java properties in Scala, I decided to flesh out the idea more fully as an open-source project. While it is not the first […]

Tagged with: scala (41), functional-programming (31), java (22), static-typing (16), dsl (1)