Do you want to use lambda expressions already today, but you are forced to use Java and a stable JRE in production? Now that's possible with Retrolambda, which will take bytecode compiled with Java 8 and convert it to run on Java 7, 6 and 5 runtimes, letting you use lambda expressions and method references on those platforms. It won't give you the improved Java 8 Collections API, but fortunately there are multiple alternative libraries which will benefit from lambda expressions.
Behind the Scenes
A couple of days ago in a café it popped into my head to find out whether somebody had made this already, but after speaking into the air, I did it myself over a weekend.
The original plan of copying the classes from OpenJDK didn't work (LambdaMetafactory
depends on some package-private classes and would have required modifications), but I figured out a better way to do it without additional runtime dependencies.
Retrolambda uses a Java agent to find out what bytecode LambdaMetafactory
generates dynamically, and saves it as class files, after which it replaces the invokedynamic
instructions to instantiate those classes directly. It also changes some private synthetic methods to be package-private, so that normal bytecode can access them without method handles.
After the conversion you'll have just a bunch of normal .class files - but with less typing.
P.S. If you hear about experiences of using Retrolambda for Android development, please leave a comment.
Congrats! That's a pretty awesome idea for early adopters...
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteHi, i'm Sergey, maintainer of HoloEverywhere library for Android.
ReplyDeleteDalvik VM works nicely with retrolambda :)
Many thanks for library and expirence.
I use next code for testing: http://pastebin.com/MrXUPn5B
Screenshot from Android Debug Bridge: http://i.imgur.com/ram0ecB.png
This comment has been removed by the author.
ReplyDeleteSo it basically let you use lambda expressiona in development for JRE5+ project using JDK8 if I understand ?
ReplyDeleteSo how is the bytecode looking ? Is it the same as if you were using a a single method interface and an anonymous class instance to simulate lambdas ?
Yes, it's pretty much the same as what an anonymous class would look like. There are a couple of wrapper methods causing indirection (so that future JVMs can in the improve the lambda implementation), but the JIT will easily inline them.
ReplyDeleteOne difference over anonymous classes is that there is an optimization (as is also in JDK 8) that if the lambda doesn't capture values from its enclosing scope, only one instance of the class is ever instantiated (it's stored in a static final field). This optimization will be included in Retrolambda 1.1.0 which I'll release later today or tomorrow.
Thanks for the explanations, I will definitively give it a try !
ReplyDeleteI've updated my comment in response to your Stack Overflow question about a backport to Java 7. I really didn't believe it could be done so efficiently!
DeleteCool. Im downloading JDK8 now :)
DeleteThis comment has been removed by the author.
ReplyDelete(This is basically a repeat of a reddit comment in case this is a better place for you to catch it.)
ReplyDeleteI'm impressed across the board -- concept, approach, code, explanation, and dealing calmly with redditors that missed the point. :)
I note that you've spent some time with Scala and are now in to Clojure. Have you heard of Perl 6, which is, very loosely speaking, in the same sort of space (as compared to the older generation of dynamic langs such as python, ruby, and the original perl)?
All of this leads me to:
Perhaps you would be willing to visit the IRC channel #perl6 on freenode [1] and chat for a while? A guy called jnthn is finishing up an initial port of Perl 6 to the JVM [2] and is digging in to java interop, multicores, optimizations, dalvik, and so on. I know they could do with pointers from someone with the level of JVM expertise you must have and I bet he'd enjoy a chat with you even if only for 15 minutes. Here's hoping...
[1] https://kiwiirc.com/client/irc.freenode.net/perl6?nick=orfjackal
[2] http://6guts.wordpress.com/2013/07/15/rakudo-jvm-news-more-tests-plus-thread-and-promise-prototypes/
I'm not using Perl, so I haven't been following the news surrounding it. (I mainly use Ruby and a little Bash for scripting.)
ReplyDeleteSure, I can chat for a while and try to help what I can. But regarding JVM language development there should be much more knowledgeable people at https://groups.google.com/forum/#!forum/jvm-languages
Esko, I just saw the exchange in the public log [1]. Thank you. :)
ReplyDeleteFyi, two leading P6ers live in Sweden and work at http://edument.se/english/ including jnthn. I've already seen enough of your work and personality to feel confident they'd be interested in you, as an employee or in their network (aiui their network is not limited to folk living in Sweden). If you're interested, I suggest the same approach, i.e. point jnthn (or masak) at this blog post, mention that I've left two comments, and go from there.
jnthn, masak, check out http://lets-code.orfjackal.net/
[1] http://irclog.perlgeek.de/perl6/2013-07-25#i_7370179
Thanks Esko, my concern is with performance. Have you benchmarked it between a Java 6 and 8 VMs?
ReplyDeleteNope. You're welcome to benchmark it yourselves (*insert here the usual warnings about microbenchmarks* [1]). Please write an article and post a link here.
ReplyDelete[1] http://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java https://code.google.com/p/caliper/wiki/JavaMicrobenchmarks
I'm guessing that the performance will be the same as with the original lambda expressions (or slightly better because of removing the indirection of method handles). The most probable cause of any performance difference is that the older JVMs don't do as many optimizations as the newer JVMs. So make sure to run all benchmarks using the same JVM version.
DeleteI agree with you.
DeleteThis comment has been removed by the author.
DeleteThat's a really Nice Development. Would like to try it out.
ReplyDeleteHi,
ReplyDeletethis really looks helpful, but I can't figure out why it doesn't work for me. Am I doing some basic mistake?
I'm using jdk8-b116 on linux ubuntu 12.04
Thanks for help
-Tomas
I have very simple HelloLambda class:
package com.test.hello;
public class HelloLambda {
public static void main(String[] args) {
process("This is Lambda", s -> System.out.println(s));
}
private static void process(String msg, Printer p) {
p.out(msg);
}
public interface Printer {
void out(T t);
}
}
java -jar -Dretrolambda.inputDir=build/classes/main -Dretrolambda.classpath=build/classes/main -Dretrolambda.bytecodeVersion=50 -Dretrolambda.outputDir=build/libs/output -javaagent:retrolambda-1.1.0.jar retrolambda-1.1.0.jar
and it produces exception:
Error! Failed to transform some classes
java.lang.RuntimeException: java.util.NoSuchElementException
at net.orfjackal.retrolambda.LambdaReifier.reifyLambdaClass(LambdaReifier.java:32)
at net.orfjackal.retrolambda.LambdaUsageBackporter$InvokeDynamicInsnConvertingMethodVisitor.backportLambda(LambdaUsageBackporter.java:107)
at net.orfjackal.retrolambda.LambdaUsageBackporter$InvokeDynamicInsnConvertingMethodVisitor.visitInvokeDynamicInsn(LambdaUsageBackporter.java:99)
at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1343)
at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:938)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:669)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:506)
at net.orfjackal.retrolambda.LambdaUsageBackporter.transform(LambdaUsageBackporter.java:28)
at net.orfjackal.retrolambda.Main$1.transform(Main.java:40)
at net.orfjackal.retrolambda.BytecodeTransformingFileVisitor.visitFile(BytecodeTransformingFileVisitor.java:25)
at net.orfjackal.retrolambda.BytecodeTransformingFileVisitor.visitFile(BytecodeTransformingFileVisitor.java:11)
at java.nio.file.Files.walkFileTree(Files.java:2667)
at java.nio.file.Files.walkFileTree(Files.java:2739)
at net.orfjackal.retrolambda.Main.main(Main.java:38)
Caused by: java.util.NoSuchElementException
at java.util.concurrent.LinkedBlockingDeque.getFirst(LinkedBlockingDeque.java:553)
at net.orfjackal.retrolambda.LambdaReifier.getLambdaFactoryMethod(LambdaReifier.java:47)
at net.orfjackal.retrolambda.LambdaReifier.reifyLambdaClass(LambdaReifier.java:29)
... 13 more
The "-jar" must be the last argument just before the jar file.
DeleteI've just released version 1.1.1 which gives a proper error message in this situation (it shows the usage help if the Java agent is not loaded).
DeleteCool it works. I haven't noticed wrong -jar option. Thank you.
ReplyDeleteMust be some stupid mistake, but I can't see it. I tried to reproduce the example for Dalvik mentioned above and caught a java.util.NoSuchElementException. I used the latest JDK8 from their site.
ReplyDeleteHere's the code I used and commands I ran - http://pastebin.com/vVbSkBBh
Thanks for reporting the problem. A recent JDK 8 had changes which made it incompatible with Retrolambda 1.1.1. I have now released Retrolambda 1.1.2 which fixes the problem.
DeleteHi Esko
ReplyDeletehaven't you tried to resolve problem of default interfaces? This is another feature that breaks our backporting from jdk 8 to 6.
-Tomas
No. I think that it would be hard to fully backport default interfaces, because they support polymorphism. One option might be to find all implementors of an interface that uses default methods, and copy the method implementation there, unless that class or its superclasses has an implementation. But I don't think that it would be worth the effort, because that alone would not give you the new JDK 8 APIs that were the reason for adding default methods in the first place.
Delete... but still I think it's great idea :)
DeleteAnd it will be useful for e.g. some API development on java 8 and re-distribute to older version. There is less code maintenance.
I think its good approach to scan superclasses and interfaces for default method, and after collecting them all, import them in 'root' class. I'm not so much in bytecode but who knows with proper motivation and time...
In short..you do pretty awesome yob!
best regards
Nice project, but unfortunately doesnt support "invokespecial" which is my main usecase...
ReplyDeleteso i started a sourcecode patching project https://github.com/wolfposd/to7andbeyond
currently its really rudimentary and only changes oneliners, but already suits my needs
I don't quite follow you that what do you mean by it not supporting invokespecial. Please give an example of Java 8 source code that doesn't work with Retrolambda.
DeletemyjButton.addActionlistener( e -> someMethod());
Deletetells me invokespecial is not supported
Now that's funny, because I have absolutely no problems doing that with with Retrolambda: https://gist.github.com/orfjackal/9879520
Deletei could narrow it down to the ant task i was building, turns out the classpath-property wasn't set correctly
DeleteThanks for this awesome tool! It works pretty well for me.
ReplyDeleteFor what it's worth (Caution: shamleless plug): I've attempted a backport of the java.util.stream API to Java 6, using your retrolambda, that, at first sigth, appears to hold the water.
https://sourceforge.net/projects/streamsupport/?source=directory
If anyone is interested: It's still in early alpha, so don't expect too much yet.
This is indeed a great news. If we can use lambda expression with its full glory in Java 6 or Java 7, then there would be lot of people who can use it, as upgrading to Java 8 is not as easy in big companies as for your fun project.
ReplyDeleteHi Esko,
ReplyDeleteI have ran " java -jar retrolambda.jar " on command line and it worked fine.Also in my eclipse I set Jdk 1.7 for running the code and jdk 1.8 for compiling the code.But when I run the code it give me this error "Exception in thread "main" java.lang.UnsupportedClassVersionError: RunnableTest : Unsupported major.minor version 52.0"
Do you think I have missed any step ? Thanks
The RunnableTest class is either missing from the list of classes to be processed by Retrolambda, or something recompiles it after it was processed.
Delete