Today begins with the big talk from the Devoxx Fr team, which sums up the story of the Paris JUG and Devoxx France.
Proud to be developer? by Pierre Pezziardi
In some social situations, when you are asked what you do in life, it’s hard to tell you’re a computer programmer. Because IT is late, costly and well, not very used in everydays life.
That’s not entirely untrue. The cost of implementing a feature is constantly growing: IT is the only industry where there is no productivity gain, the opposite is true! Our industry is evolving very rapidly, we are programming in languages that didn’t exist 5 years ago. And yet, some things never change. Project cycles, despite agility, are long. Moreover, there’s a clear distinction between roles, most notably between thinkers and doers. Agility was meant to address that.
Our method was to develop integrated products, and that meant our process had to be integrated and collaborative.
Yet, though Steve Jobs formalized that, though productivity decreases, though agility works, not much changes. There’s a block somewhere and this block comes from the people themselves, that resist change. Evolving from a planned economy to agility is very hard.
We could stop at that, and tell us we cannot do nothing but in fact, we can do better, not changing overnight but step by step. As an example, at BRED, customer service was handling more that +120k calls per year (80% support). 10k calls were about Citrix session freezing: adding a simple button on the screen helped reduce the this number by 8k. Another way to reduce was to change a simple error message label, from "Technical error" to "Your account is under watch, please contact your local counseler".
IT is misunderstood by people outside IT. For example, the notion of technical debt is invisible to top-level managers. In order to communicate with people outside your culture, you have to talk their language. It’s done through graphics and semantics. To go beyond productivity decreases, we’ll have to drastically change our culture, but in the meantime, we can act in small increments with added value.
Another thought is that IT is aligned with its organization: bureaucratics organizations produce bureaucratics IT. For example, Leman Brothers implemented the Bale 2 standards (risk management), and yet they gained nothing by it (you remember it?). In essence, understand not only IT complexity, but also remember IT is a support function, and has to bring business value. Examples of such added value systems are Wikipedia, Google, and so on. What do these tools have in common?
The simple, poor, transparent tool is a humble servant; the elaborate, complex, secret is an arrogant master.
In essence, those tools are based on everything is possible but instead of standard enterprise policies where everything is forbidden but. The bid is on trust toward users. The approach is to empower users, let users do mistakes, but be able to correct them easily.
It may seem that we as computer programmers cannot do something about it (we only are developers), but in order to change the system, we have to change ourselves first. This is not a recent strategy, though: Gerald Weinberg describes it in his book The Psychology of Computer Programming.
In conclusion, we are working for organizations that segregate people. In the IT, we reproduce the system: programmers, project managers, business analysts. Our duty as programmers (and people) is to break those barriers, and talk people outside IT.
This could be Heaven or this could be Hell by Ben Evans and Martijn Verburg
The talk is about the future of Java. There are two possible view of the future, Heaven and Hell.
In the Heaven view of things, Java 8 comes out in 2013 (as promised), it remains the #1 language and Jigsaw (Java’s promised modularity) is largely adopted. Java 9 brings out reified generics 🙂
In essence, we are in the middle of a changing environment:
- hardware is definitely multicore
- mobile is here to stay
- Emerging markets are going to bring many more new Java developers (millions are expected)
Open Source Software becomes really part of the mainstream and everybody (the general public) understand the notions behind it. On the social networks side, they observe strict confidentiality policies. As a corollary, configuration for controlling those parameters is easy and readily available.
In the Hell view of things, Java 8 is not released until 2015 because the community fails to test it enough; Jigsaw is never made available. Java 9 is not released until after 2020: it’s no more #1, developers flee to the .Net platform and alternative languages die.
On the hardware side, it becomes even more fragmented. Worse, patent litigations dominate the market and OpenJDK cannot manage both. On the mobile side, Android and Java stay separated and Android fragmentation goes even further.
In Hell, millions of new developers turn toward .Net and VB is taught in school. Apple and their look-alike continue to promote lock-in. Competitors see the benefits of this approach and take the same way. For Open Source, OSS developers become elitist and many new ideas never concretize. Actual startups, as market leaders, are worried about new nimbler competitors and start heavily lobbying, innovation is smothered.
Facebook and its successors dominate the Internet and privacy is completely lost. You log in with your Facebook at work and your boss can watch what you do the all day.
In conclusion, the future can be anyhting, but whatever it is, it’s up to us.
Spring is dead, long live Spring by Gildas Cuisinier
Guess what, this talk is about Spring and more precisely the war between Spring and JavaEE.
Episode 1
How Spring appeared? The JCP made a platform proposal years ago, that went by the name of J2EE (at the time). There were some problems: migration from a standard J2EE platform to another one was not as easy as announced and tests were coupled to the platform (no unit tests). Rod Johnson created in response an Open Source lightweight framework to address those problems: Spring was born.
Second evolution
At first, initial configuration was done through XML (there wasn’t so many boos against XML at the time).
At its inception, a single configuration was necessary.
Then, the import
command made possible to have multiple neatly separated configuration files.
Third-party frameworks made their way into the framework, and for some, XML configuration was a nightmare (Acegi for example).
Spring evolved with specific namespaces to made configuration for those frameworks easier, it’s something akin to DSL for XML.
JavaEE 6 sexy again
With Java 5 bringing annotations, Spring continued its evolution and took them in. We still need a little XML configuration to scan for annotations, though. Meanwhile, the JCP learned of Spring’s successes and delivered JavaEE 6: it’s simple, testable and lightweight, all is achieved through annotations. Yet, migrating to JavaEE 6 is a barrier to acceptance.
Also inspired from its competition, Spring 3 brought full-annotation configuration, XML can be directly discarded.
JavaEE in 2012 from a survey
Results from http://cyg.be/SpringJEE show the following results: more than half of installed JavaEE are version 5. A majority of not-yet adopters think about migrating in less than a year. More than half of Spring users do not consider migrating to JavaEE. On the contrary, new users are very fragmented: only a quarter think about JavaEE 6, while another quarter think about Spring.
On the Spring side, results show a quicker migration to newer versions. For example, half of current Spring users use v3.0 while it was delivered at the same time as JavaEE 6 and around 20% use Spring 3.1. It’s mostly because migrating is easy (while JavaEE migrations require other application servers).
Episode 3
Spring 3.0 brought the JavaConfig
approach, a no-XML configuration but didn’t entirely close the gap, Spring 3.1 does.
For example, component scanning can be done through @ComponentScan
, scheduling can be achieved through @EnableScheduling
and MVC is provided though @EnableMVC
.
What these annotations do is easily understood from the underlying code (it was not so easy in XML).
A great feature of Spring is an abstraction over cache through @Cacheable
; it provides no implementation however so you can use different implementations (ConcurrentHashMap, EhCache and GemFire) under the cover.
The feature is also available through XML.
Spring 3.1 also provides:
- Hibernate 4 support
- A new XML
c:
namespace for constructors in XML - JPA use without persistence.xml
- miscellaneous improvements in Spring MVC
Conclusion
In conclusion, Spring is not only the Spring framework, but a whole ecosystem is readily available: Spring Data, Spring Batch, Spring Mobile, Spring Security, Spring Android, Spring Flex (ok, let’s forget this one), and many more.
IMHO, too much defending Spring at the beginning which left not enough time to deal with real stuff at the end. It’s not a good sign when one is on the defensive.
Java concurrency from the trenches by Alex Snaps
Having learned yesterday that we should prepare for concurrency, I decided to attend this talk in place of Client/Server Apps with Java and HTML5 [I couldn’t reproduce the code snippets].
The goal of this talk is to have a better understanding of concurrent code, do more with java.util.concurrent
and finally make best use of CPU cores.
What’s concurrency anyway? And what about threads, locks and actors? The main thing about concurrency is state and how to share it. A thread safe is a class that obeys its contract, whether by a single or multiple threads, whatever the scheduling and without external configuration.
- Atomicity: no threads should expose intermediate or transient state that violate post-conditions of a class
- Visibility: in a multithreaded environment, there’s no guarantee I can read back a value from a variable just after having written it
Taking Hibernate has an example, there are multiple statistics to track: entities and queries counters, max duration information, etc.
public interface Statistics {
public long getQueryExecutionMaxTime();
public long getQueryExecutionMaxTimeQueryString();
}
This interface (no implementation there) cannot guarantee its contract, cause the two methods are linked but it’s not possible to easily enforce this.
- In order to get thread safety, a naive implementation would be to put the
synchronize
keyword in front of each method. The drawback is that a single monitor for each method has a huge impact on performance. - The next step is to have different monitor for each variable that is accessed, and to use the same monitor for the same variable across different methods with
synchronized
blocks inside the method. - Java 5 brings the notion of read/write locks: there’s still only a single access for writing but it allows multiple concurrent read access. For statistics, there’s no interest since there are many writes but only a few read.
- Also in Java 5, we have
Atomic
classes that can be used for synchronized variables. Using such atomics, tests are even slower… even though I didn’t catch why.
Conclusion, finer grained locks can improve performance (i.e. adapt your lock granularity to your needs and context).
Better yet, there’s a no-lock solution, by creating a single State
object.
As an alternative, the volatile
keyword can be used to guarantee the read value is up to date when accessed.
Finally, another solution could be Compare-and-Swap.
CAS is all about not enforcing thread-safety, but rather preparing for failure and dealing with it.
In conclusion, aim for non-blocking strategies and only try and use locks when required.
I must admit I didn’t get the most of the session, it was full of content and I probably lacked some of the background skills. Slides on Parleys will probably help me go deeper into my understanding because there was a lot of stuff.
DevOps: not only to manage servers by Jerome Bernard
Following my (re)discovery of virtualization, I thought the session could be very insightful.
The session is a feedback on how the speaker had to organize insider teaching sessions, while minimizing impact on infrastructure. The biggest constraint was delay (less than two weeks from bare machines).
Solutions that were rejected included:
- Ghosting strategies (a single master redeployed on client machines before each session)
- Complete reinstallation from the local network through scripts
- Account creation/destruction for sessions
The final solution is a Chef server, with cookbooks being polled by Chef clients deploying on each client’s VirtualBox. This let attendees get full admin rights on their guest system (while locking the host system).
Chef let us automate installations through recipes and cookbooks. Chef polls every so often so that configuration stays in sync on each client. Chef’s advantage over Puppet is a bunch of recipes ready to be used. Finally, Chef provides an administration console that let us follow clients state. On the physical machines, a Chef server was installed to only automate VirtualBox installatiobn and Vagrant files. Also, effort was put to optimize network usage.
VeeWee was born as a Vagrant extension but now has a life on its own. It let use Vagrant masters easily, through templating. During the normal course of things, there’s no need since they are plenty of Vagrant baseboxes available. In this case, the need was to change to the French locale from the English based master. VeeWee launches a VirtuaBox and simulates user clicks on it. As a side note, VeeWee recently made Windows masters available.
Master definition files were edited to achieve the previous steps:
- definition.rb: changed the used ISO, as well as the MD5 hash
- preseed.cfg: changed locale to French
- postinstall.sh: removed Puppet installation
Once VeeWee finished building the image, a single command was used to create a Vagrant box.
Vagrant let us create a VirtualBox image from the CLI, through a single text-based Vagrantfile. It can easily be integrated with Chef and Puppet. Note: we can do many other things (network management, port forwarding between VM and physical machine, directory sharing between host and guest, multiple VMs management (and their network configuration), …). In the case study, the Vagrantfile was dynamic (updated by Chef) so that the number of allocated CPUs were the number available on the physical machine, and so on. The Chef recipe downloaded Java, Maven and MySQL.
On the host systems, Chef solo was also installed. Vagrant and Ched only installed the softwares necessary to a particular session: Eclipse (as a .tar.gz), native packages (SVN) and custom symbolic links.
Feedbacks:
- Chef
- DNS resolution is very important
- The official Java Chef recipe doesn’t work anymore since Oracle now requires JavaScript and cookies to download from the site. Better get the JDK once and make it available on an internal web server
- Packaging the Android SDK is not easy, it’s only an installer
- Vagrant
- Test on your notebook before deploying
- Vagrant provision/reload to update a Vagrant-managed box
- Vagrant and VeeWee should be now more compatible
- VeeWee
- Keep in synch VirtualBox versions and VirtualBox masters version
Finally, other uses came by: developer machines are provisioned by the same system and user-acceptance environments are managed by Vagrant. A good idea would be to use Continuous Integration of VMs, so that bad surprises don’t happen at the last moment.
All in all, another session full of content that will need to be reviewed once or twice to get the most of it.
Behind the scens of day to day development at Google by Petra Cross
The talk is about teams and roles, development workflows and those used at Google.
Teams and roles
There are +10k developers across +4O offices checking in every minute. At Google, roles are defined with clear-cut responsibilities: engineering directors (ENG DIR), product managers (PM), engineering managers (ENG MGR), software engineer/tech lead (SWE/TL), software engineers (SWE), software engineers in test (SET) and test engineers (TE). The team hierarchy has no more than three levels. Teams are created based on needs, for a project, not for here forever.
Google’s way of achieving its goal (organizing world’s information…) is by creating software. A feature developement goes like this:
Idea > Features > Planned > Worked On > In Code review > Tested > Canary > Live!
Note that "worked on" include unit tests, so that code review can get you a write more tests result.
Everything that’s it in the continuous integration goes into a release. Google has a eat your own dogfood policy so that Google employees can give feedback. Then, if everything goes right, the release is pushed to the Canary: it spreads to only a fraction of the users, depending on the particular product.
How to reduce development time? Either add resources or reduce waste, Google’s way is to reduce waste.
Development workflow
Three methods are relatively well-known:
- Waterfall is old school and come from old manufacturing industries. No need to go further…
- Spiral is the next step; it’s a sequence of waterfall, each producing a prototype of a product. The iteration cycle is about 6 months.
- Agility is the newest approach. It’s all about customer collaboration, having feedback and adapting for changes. There are different flavors: XP, Scrum and Kanban (among others)
Google workflow
Google does mostly a bit of everything but mostly Agile. Ten things Google found to be true first lesson is that when you focus on users, everything else falls into place. Customers think in User stories while Devs thinks in tasks.
Example user story: "As an ATM user, I want to be able to view my balance". Possible tasks include:
- Define an API which the ATM will be able to talk to the backend
- Implement the backend
- Add the GUI
In Kanban, a task can be in four different states: ICEBOX, BACKLOG, CURRENT (or WORKING ON) and DONE AND VERIFIED. Tasks in the icebox may never get done, so constant cleaning of the icebox is necessary. The backlog should have tasks for one to one-and-half iteration to guarantee noone runs out of work. Tasks in the backlog are things everyone agreed to work on; they are picked by developers. Everyone having a visibility on the backlog renders daily meetings useless because when picking a task, passing it in the CURRENT state, you put your name on it. On whiteboards, there’s a single swimlane for each state (save ICEBOX) and tasks are writtten on post-it. When passing in the CURRENT swimlane, you put your name (and perhaps picture) next to it. You can also pair program, it’s irrelevant to the workflow.
As said before, daily standups are optional but weekly team meetings are mandatory as well as monthly retrospective meetings (see later). In weekly meeting, you estimate tasks from top to bottom, by planning poker. When a consensus is not reached, lowest and highest estimators have to explain their reasons. An important point is that there’s no multi-directional discussion in order to avoid involving emotions. Then you re-estimate until a majority agrees. Estimation is done in points, not in time, because estimating in absolute is hard: it’s easier to tell that a task will be done quicker (or slower) than the other one. Likewise, velocity is computed in points: when a PM asks for a feature, it’s broken down into tasks and the engineering manager know if it can be achieved given his/her team velocity.
When stuck, developers are expected to learn from materials and ask questions so as to avoid island of knowledge situations. Besides, managers and tech lead don’t care how you achieve your task, only that the task is done. |
There’s also an emphasis on retrospective once a month. Everyone expresses: what went well, what went not so well and what could be done better.
Ok, there’s no magic involved… A bit disappointed, because I was probably expecting to have life explained to me. Seems Google manages its projects like everyone else, the success must come from other factors: culture (though I don’t see how I could apply the "get stuck and learn from it" approach), editor approach, or something else I cannot point my finger on.