Part 1: The rant about a wrong example

I was meaning to write about upcoming Scala features but got sidetracked by work and GeeCON related stuff this week. I’ve just read a post about Dynamic on scala.net.pl and… It’s totally wrong! :< C’mon people – you’ve got scala _as_ your domain name, so at least run the examples in a REPL before you post something ;-)

Ok, so on the REPL – the first line will compile. Dynamic is in fact a new Trait in Scala 2.10 and you can create new instances of Traits (just as you could in Java with anonymous inner classes) using the new Trait {} syntax. Line 2 though, it utterly wrong. It will cause 2 compile time errors, namely:

Dynamic is NOT “a magic type you can set random stuff on”, it’s a bit more discrete. You can build such construct using Dynamic – sure, but that’s not all it is.

Part 2: A deep dive into Scala’s Dynamic type

Ok, end of rant and back to the basics. So… How do we use Dynamic? In fact, it’s used by implementing a few “magic” methods:

• applyDynamic
• applyDynamicNamed
• selectDynamic
• updateDynamic

Let’s take a look (with examples, at each of them. We’ll start with  the most “typical one”, and move on to those which would allow the construct shown above (which didn’t (back then) compile) and make it work this time ;-)

applyDynamic

Ok, our first magic method looks like this:

So the signature of applyDynamic takes the method name and it’s arguments. So obviously we’d have to access them by their order. Very nice for building up some strings etc. Our implementation will only print what we want to know about the method being called. Did it really get the values/method name we would exect? You can try it out (it’s copy paste ready) in your Scala 2.10.M3 REPL – if you don’t have one at hand: yeah, the output would be:

  methodName: dynamicMethod,
  args: with,some,1337

By the way, did you know about RichString’s stripMargin? It’s a nice way to still have nicely printable strings when you’re using multiline strings. By default it trims everything that is before the | sign (from the left side, for each line). The sign can be overriden of course if you fancy $ signs for example… ;-)

applyDynamicNamed

Ok, that was easy. But it didn’t give us too much control over the names of the parameters.
Wouldn’t it be nice if we could just write JSON.node(nickname = “ktoso”)? Well… turns out we can!

So this time instead of just a list of values, we also get their names. Thanks to this the response for this example will be:

Creating a node, for:
 "nickname": "ktoso"

I can easily imagine some pretty slick DLSs being built around this!

selectDynamic

Not it’s time for the more “unusual” methods. apply methods we’re pretty easy to understand. It’s just a method with some arbitrary name. But hey, isn’t almost everything in scala a method – or we can have a method on an object that would act as a field? Yeah, so let’s give it a try! We’ll use the example with applyDynamic here, and try to act like it has a method without ():

Hey! Why didn’t this work with applyDynamic? Yeah, you figured it out already I guess. Such methods (without ()) are treated special, as they would usualy represent fields for example. applyDynamic won’t trigger on such calls. Let’s add selectDynamic to the mix then, shall we?

And this time when we execute HasStuff.bananas we’ll get “I have bananas!” as expected. Notice that here we return a value instead of printing it. It’s because it “acts as a field” this time around. But we could also return things (of arbitrary types) from any other method described here (applyDynamic could return the string instead of printing it).

updateDynamic

What’s left you ask? Ask yourself the following question then: “Since I can act like a Dynamic object has some value in some field… What else should I be able to do with it?” The answer is obviously: “set it”! That’s what updateDynamic is used for. There is one special rule about updateDynamic though – it’s only valid if you also took care about selectDynamic – that’s why in the first example the code generated errors about both – select and update. For example if we’d implement only updateDynamic, we would get an error that selectDynamic was not implemented and it wouldn’t compile anyway. It makes sense in terms of plain semantics if you think about it.

When we’re done with this example, we can actually make the (wrong) code from the first code snippet work. The bellow snippet will be an implementation of what was shown on the first snippet on that other website, and this time it’ll actually work ;-)

Using this Dynamic “MagicBox” we can store items at arbitrary “fields” (well, they do seem like fields, even though they are not ;-)). An example run might look like:

scala> MagicBox.banana = "banana"
MagicBox.banana: Any = banana

scala> MagicBox.banana
res7: Any = banana

scala> MagicBox.unknown
java.util.NoSuchElementException: key not found: unknown

By the way… are you curious how Dynamic [source code] is implemented? The fun part here is that the trait Dynamic, does absolutely nothing by itself – it’s “empty”, just a marker interface, like in the good ol’ Serializable days ;-) Obviously all the heavylifting is done by the compiler here.

Anyway, thanks for reading and… don’t overuse Dynamic, will ya? :-)

There is a land in which API calls can not always be trusted. A land where docs lie you in the face about return guarantees. It is here, where the Lords of Code join hands – and with their Keyboards of Steel smite all evil in search of the hidden truth… Here be dragons – API weirdness.

– Author unknown, 2012

That being said, I got a bit sad today the reason being: It’s actually really HARD (impossible?) to get your phone number on android. Yeah, you can:

But… It won’t always work. After some googling around I found an issue related to this on the android code.google.com. Sadly, it’s explained there “black on white” that if the simcard doesn’t contain this info (does not have to) you’re basically a bit screwed, and won’t be able to obtain the phone number (I did a bit of research and it really seems impossible – if you know a way, please let me know :-)).

The SIM in the device has a slot for the phone number, but it isn’t required for operation. Some carriers may choose not to populate this SIM field. [...]

That’s v. interesting (to me, not a GSM-network-operator pro ;-)), so the phone number is actually assigned by the network when the simcard comes talking to it. True, that makes sense – you can switch SIM cards and still keep the same number you had all the time… What to do? What to do?! I still need a way to find out the number, and sadly when looking at sms messages already stored on the phone, they only contain an “address” field which is the “sent from” number.

The outcome of this story is the interesting part here I think. I switched to asking the user for his phone number – it’s really “good enough” for my use case :-) I do something quite funny, the phone.number is actually a lazy value that, if it needs to, will create a dialog to ask the user about it.

So in the activity we add a listener on the Future[String] (actually that’s Guava’s SettableFuture[String]), so we’ll update the UI when the value gets updated.

Btw, yeah I have some implicits for Runnable creation etc. So the first parameter here is actually an implementation of Java’s Runnable. Moving on, let’s take a look at the implementation of phone.number:

So we still try to use the TelephonyManager to obtain the phone number – sometimes it works. But if it didn’t, we fallback to asking the user about his phone number. Here’s the implementation of askUserForHisNumber():

It’s explained in-line there, so take a moment to digest it. The general idea here is to NOT block the main UI thread, while allowing the dialog to show up “whenever it needs to”. In this case it’ll show up when first needed – when someone gets the Future. In the askUserForHisNumber implementation we store the once entered phone number, so it’s reused from the shared preferences on the next app start instead of asking the user again.

Also notice that using this flow we created a quite nice UX (IMHO), as the user doesn’t have to crawl to the settings activity to set this – he’ll be asked for the number if it’s needed, right away :-) If you’re interested in more Android+Scala posts, just drop a comment – I’ll happily follow up in a blogpost.

The syntax to use interpolated strings looks like this:

And it’s actually just a shorthand for the bellow:

So the $name placeholder was replaced with an empty “part of a string” in the StringContext, and the f can be seen a bit similiar to RichString#format() here, it puts the values in their places. The interesting trick here is that the logic behind this trailing “” string is that f() inserts the given parameters between the parts given to StringContext – this way it doesn’t have to do any repetetive parsing etc at runtime, it just inserts into “known places” which is a better (performance wise) idea than keeping the pattern with dollar signs and parsing it each time. :-)

Another nice feature in contrast to (for example) Ruby’s string interpolation here is that “value not found” problems can be detected at compile time, so the bellow example would not compile:

“Typesafe!” one might want to shout… ;-) Anyway, I think it’s a welcome feature and will improve our loggers from:

to a nicer

I’ll also cover the “more” interesting features in some of my upcoming blogposts, so keep an eye out for posts about: Dynamic, Macros and Manifests vs. TypeTokens :-)

et cetera… One of those less useful Words I have implemented is the NotAdverb. It’s allows you to negate Booleans using a not method that’s being pimped into them.

So it obviously doesn’t provide any more value than just using the ! operator. I guess it may be nicer to read, but that’s it… A colegue of mine pointed out that, while perhaps it may be nicer to read… is the runtime overhead of this implicit conversion REALLY worth it? To anwser this question I decided to see for myself (out of curiosity) how much such an implicit costs (in terms of nanoseconds).

I’ve recently discovered Google Caliper which is a very small micro benchmarking framework, so I decided to use it. Sadly it turns out it’ll throw

Class [pl.project13.implicitoverhead.Benchmarks$JustBoolean] has no parameterless constructor.

even though my test classes (written in scala) obviously did have what Scala considers as parameterless constructor. I almost went out to fix the problem in Caliper, when I found that Daniel Sobral already went out and fixed Caliper to work with Scala. So here’s the full implementation of the benchmarks I have written:

The settings I ran the tests on are as follows (default settings would be a bit too small I figured for such simple tests):

repetitions = 3000
warmup time = 3000 ms
test time = 10_000 ms
trials = 10

Sadly I was not able to run caliper with –measureMemory, which would have been really interesting too I think. I didn’t spent too much time with it, so maybe it ain’t so hard to include the javaagent it requires hm hm. Anyway, here are the results, first in command line format and then in all it’s uploaded glory:

And then the graphical representation of it (available on http://microbenchmarks.appspot.com/run/ktosopl@gmail.com/org.example.Benchmark).

It’s quite obvious that the method with implicits would be slower, but it’s still interesting to first MEASURE (!) and then judge some solution. So here we’ve got an average of 300% more slowness than just using the ! operator… Although we’re still in very small nano second numbers, that can be considered quite an overhead – depending on what your use-case is. In our case it isn’t really the “slow downer” as we do more network operations etc, but all in all I think we’ll say goodbye to the NotAdverb ;-)

By the way, since caliper also automatically uploads your test results online, it’s a really cool tool to show team members how your implementation is faster than theirs etc ;-) I recommend you to check it out, even if it’s still in it’s early stages… :-)

The JVM is awesome.

But back to the topic of this post: We’ll see how you can leverage both Scala and JRuby in one project. Each will doing what it does best:

• Ruby will handle the FAST frontend development as one of Ruby’s strong points is obviously Rails which made it so popular. Though there are people implementing APIs in ruby, if you need pure performance, you may need some other lang (vide airbreak, twitter…).
• Scala will be doing all the heavy lifting including some of our existing code which interacts with zookeeper or uses Akka actors to crunch some numbers.

Another great point here is the reuse of our existing scala codebase which has some nice libraries we’ve implemented during our day-to-day. Anyway, let’s see some code!

Start out with a plain Rails 3.2 project

You know the drill. rvm, ruby, rails new. The only difference here is that we’ll use JRuby right from the start here, so:

JRubify it!

Next we’ll add some more dependencies to our Gemfile:

Most important here is the jruby-scala gem, which acts as a bridge to allow the use of scala constructs in ruby, more naturally. For a longre read on this check out Daniel Spiewak’s blogposts about it.
Now do the usual bundle install (be sure that you’re running jruby (!)). If you don’t want to rvm use jruby, you could also run bundler like this: jruby -S bundle install.

One thing I’m not covering in this blogpost is how to throw out ActiveRecord from rails. But that’s easily googlable: Just don’t require it, as shown here: Rails 3 without ActiveRecord. Of course you CAN use active record with JRuby etc, but in my project we won’t need it, so I’m removing it.

Scalify it!

Ok, now that we have all the gems we need let’s proceed with putting scala into our lib/ directory. We’ll need it at runtime obviously… :-)

One way to include jar dependencies into your project is by simply pasting them into the lib/ directory, so if you need some cool-java-library.jar, just drop it in there! It’ll also be automatically packaged with the app into the warfile by warbler – yay!

Require ‘java’!

Kk, now on to some code, let’s assume I have a nice “number formatter” I’d like to use in my rails app, that I’ve already implemented in some “commons” library in Scala. (Obviously that’s not the only thing from scala we’re using in our app but it’s just an example class ;-)). Here’s how we’d write our controller:

By the way, PrettyOrdinalNumber is a trait in scala. There we use it via implicit conversions to nicely print numbers when counting servers etc ;-) A small thing but nice to have.

The view here is boring – we just print this one variable; So I won’t even put a code block here for it ;-)

Run it!

We still have not launched the app… Does it work at all? It does – but let’s check. The nice thing here is… You can still simply run the app via the good ol’…

command. (Just be sure you're running JRuby.) Tests and everything just works as you'd expect it to - no worries here.

One thing worth mentioning here is that currently assets pipeline may not be working with JRuby (or maybe it does but I didn't care so I just generate assets before I build the war file). So you'll want to disable it in production like this (in config/enrivoments/production.rb):

Then in order to pre-generate assets (compile scss -> css, et cetera)  you can run:

Which will generate all assets and throw them into public/assets/. Oh and don't panic if this takes a long time - yup, sometimes it does.

Ship it!

Now that we've got our app running let's see how we can make a war from it... We'll be using the (awesome) warbler gem here. So if you don't want to tweak anything, you just run:

and it'll create a war containing all your libraries from lib/ and obviously the entire rails app. (Warbler is "smart" and can detect if your project is a rails app etc). If you need more control over what warbler is including (and where) in your warfile you can run:

Which will create a warble.rb file in conf/. I'll leave it up to you if you need more tweaking or not (you could exclude temp/ and other directories from the warfile for example).

So... We've got our shiny war file. How can we run it? I'll show you two ways of dealing with this, first: on Glassfish 3 (a applicationserver) and on Jetty (a small and fast servlet container).

So, with glassfish installed you could just:

You may want to set it's context path to / instead of /scala-on-rails but as you can see, it started and you're running JRuby... accessing Scala code, on a normal JEE server!

If you prefer a smaller server, which ain't so easy to administer remotely (asadmin rocks) but is a bit smaller etc, you should check out Jetty. There is a gem that claims to run apps on jetty "just so", but it didn't work out for me. And all in all... Deploying to jetty is also just 1 line of code, so why would a need a gem for that? ;-) Here's how to deploy your war to jetty:

...and that's it! Look for the app on http://localhost:8080/

Ruby <3 Scala <3 Java <3 JVM => Best friends

That's it - we've deployed our first app using JRuby and Scala libraries. Such setup is doing pretty well for us and we can gain from scala/java power inside our quickly developed rails app.

The only tradeoff we made here is the fact that some gems require native extensions, which won't work with JRuby. But we gained a hell lot of powerful libraries (lessening code duplication among projects in the process too!) and "1 file deployments", which are really a lot nicer than deploying rails apps (that's a personal opinion there, don't kill me ;-)).

That's it folks. All hail to the JVM and keep on hakking!

One bad / weird thing I noticed in it though was the fact it didn’t understand the “jump by word” shortcuts I’ve grown used to in the bash prompt (alt + left / alt + right).

Thank god today I’ve found out how to fix this: Open up iterm preferences and then go to the Keys tab and add such bindings in the global key bindings:

The article is about different young devs getting into the Java world and trying to “move it” (GeeCON pun intented). I for example facilitate a bunch of meetings in Poland, like Code Retreats, JUG meetings (as part of the cool PolishJUG, or meetups like Android hackathons or reading clubs with friends: Software Craftsmanship Kraków… There’s more, but that’s the community part of it ;-) Speaking and doing conferences is yet another long story… :-) I’m quite happy to see oracle reaching out to people and putting life back into Java communities, I mean – the JVM is far from dead at the moment, so let’s enjoy the upcoming interesting times! :-)

I really like the word play they did using my head and the Java Magazine’s slogan on the title page, turning it from “By and for the Java Community” into “B …. the Java Community” – nicely played! :-)

Needless to say, I’m really proud of it and will continue to do my pest for our Java communities in Poland :-) By the way, I hope you’ve already registered for this year’s GeeCON, right? :-) We’ll have a bunch of cool stuff prepared for everyone… :-) If you want to read the whole issue you can download if for free from oracle’s site: http://www.oraclejavamagazine-digital.com/javamagazine/current, there’s a nice article about the distruptor pattern on the last pages so you might want to take a look at the whole issue, not just the bellow image ;-)

In other cool news this week: I’m currently hanging out and hacking around in Mountain View, California, which in turn is quite near San Francisco:

Awesome! :-)

In our example we’ll want to unmarshall a Cookie object from it’s string representation, a pretty typical task. In order to do that (for example with JAX-B), we’ll need a Class object we can pass to the deserializer – so it can work all the field mappings etc. Obviously the second example in Java just isn’t enough – we need the Class object, and just using <Cookie> isn’t enough to retain this type information. Scala obviously suffers from the exact same problem as it’s a JVM issue (type erasure, grrr!).

But even so, the above Scala code will work fine if we do some tricks in the method definition. We’ll resort to using:

• a second (implicit) argument list
• Manifest[T]

That’s a bunch of scary looking terms, but it’s actually quite easy. Let’s take a look at the definition of the Scala version of unmarshall.

First, let’s analyze the signature of this method. Obviously it’s parameterized method, thus we need the [T] in front of it. Next is the argument list, with just one argument – str, the string representation we want to unmarshall – easy. Now for the more fun part – the second argument list. Yeah, you can have multiple argument lists in Scala. “Why?”, you ask? For example in our case the second list is implicit – this means that the compiler will figure out and try to fill in these arguments with matching types it finds in scope. In the case of supplying a Manifest[T] it’s fairly easy – it will just create the manifest based on the T type, during runtime at the call site of the method, and pass it into unmarshall.

A Manifest[T] can be thought of like a “better” a Class[_] which contains full type information – so for example we wouldn’t loose the information contained in a List<Cookie> like we would with plain Java (where we would have to hack around this JVM issue via a TypeToken). Anyway, thanks to the Manifest we’re able to get the Class we need to get our unmarshalling done! Manifest.erasure contains the type we need. If you want to read more about Manifest, the ScalaDoc about it does a really nice job explaining how to use it.

— EXTRA for Scala Geeks —

You may have noticed the word “implicitly” appearing from time to time in Scala discussions. It’s a shorthand for accessing shorthand passed implicits. Let me explain this on our example – so instead of the code above we could do the following (which does the same thing):

So instead of creating a second argument list we say that the Type parameter T will be used as a Manifest. And then we can access it via the implicitly magic method. This approach has been made possible since Scala 2.8, due to how often the above pattern was discovered in real life code.
Anyway – it’s up to you which version you like the most.

The case is this: We have a common configuration (a bunch of scala objects) and some helper framework to help us in writing unit tests that integrate with the model. And then there’s the model obviously. If you’d like to allow people to use this test scoped helper framework, if should be in another project (which using maven conventions would usually be called something like “model-testing”) and it would be used as a dependency in the maven project which wanted to test some model related behaviours. The problem with maven here is that it would require us to put those helpers into a new project, and also… put the configuration scala objects into another projects – as they’d need to be shared among the helper utils, the model and all projects using both of them (the details about “why?” are a bit specific to the project, but depending on how it’s structured and how it should work that would be the way to go with maven).

Thank goodness we weren’t unaware of the excticing strengths that sbtr gives us, and instead of creating multiple projects we were able to got away with just exposing the test sources as classpath dependencies for the projects which would need the model. This can be done via:

All the magic is in the second-to-last line. It’s something sbt calls “per-configuration classpath dependencies“. And if explained in layman terms means “i want the classpath used in the lefthand project in the scope X to be used in the right hand project in scope Y“. So in our example we define that we depend on the model project (a Project instance), and declare that it’s compile classpath should be included in OUR compile classpath, and it’s test classpath should be included in our test classpath. You could do the same with “it” (for integration tests) for example… or for any other (so called) “Configuration” defined within sbt.

Using this we were able to avoid creating a “bazzilion” (a number which you can’t count to) of projects to achieve a simple goal – share test helpers. I’m getting more and more excicted about Scala and it’s tooling. As you can see SBT enables you to do some quite expressive and powerful things which would be impossible to do in maven for example. If there is any downside to SBT then it’s that the documentation is a bit chaotic, but the guys are really working on that, so I’m full of hope and optimism about it. :-)

As from what I’ve seen a lot of android apps still get written without dependency injection – which saddens me – static global variables or weird helpers still come up in apps I do security / performance / code quality audits for – so I thought that’s something I need to teach the general audience.

The conference is a 2 day event. Organized by 2 students from Politechnika Krakowska (and a few of their helpers of course), and it’s a free event. We, as Lunar Logic Polska, were proud to be one of the main sponsors of the event. I also personally helped out with a few tips etc, as part of the PolishJUG and KrakówGTUG. We also got some swag from the polish Google office which was a really nice gesture from them… :-) But back to the presentation! Here it is:

You can actually use RIGHT/LEFT and H controls in the above iframe to go through the presentation! Sources for it are available on my github.

From the audiences response etc, it seems that this one has been the cleanest, in the sense of “most fluent” presentation I’ve done up until today. I’ve also went into the code and showed off some minor details. In Intellij IDEA 11 of course – be sure to check it out if you’re an Android (or any kind of) developer! The productivity boost it gives you is “zomg wooot!” ;-)

I also really enjoyed the talks I was able to attend, and met some old (and possibly quite a few new?) friends. Hope to see you all again!

UPDATE!

This post has been awesomely mentioned by @roboguice :-)

UPDATE 2!

An awesome insider tip from within LLP I recieved today:

One guy told me that he was on an Android presentation by some long-haired guy and that it was really good, way better than the rest ;)