Friday, September 25, 2009

Embedding web UI components in Eclipse using e4

The demand for software that can be accessed through a web UI is increasing. We already expect software for occasional use (e.g. online shopping, banking, filling out forms, ...) to be accessible from a browser, without having to install anything locally. Over the last few years, more and more producticity applications (email, documents, spreadsheets, etc.) have become available through browsers as well. It is only a question of time until software development tools (at least for a certain class of uses) will have to have a web UI as well. In fact, a few browser-based IDEs already exist, such as Mozilla's Bespin project, and a very recent example: embed, a special-purpose IDE for embedded development.

At the same time, locally installed "rich client" applications, especially software development tools, aren't going to go away soon - there is a huge investment in the tools we have today, and I don't expect that all of this will be reimplemented using other technology just because web UIs are now the cool new thing to do.

So the obvious question is, how can we, the Eclipse community, prepare ourselves for the trend towards the web, while keeping our desktop users happy?

One approach is to keep the current programming model and just "make the stuff run in a browser". Fundamentally, this is very hard because the existing code we have today is not written with a client-server separation in mind. To a certain extent, it can be made to work though, and we have two examples of this at Eclipse, in form of the RAP project, and the experimental "SWT Browser Edition".

The basic idea behind RAP is to place all application and UI logic on the server, including all widgets and their state, and to render those widgets on the client in a web browser while sending user events back to the server. However, there are a couple of characteristics of RAP that limit its applicability: For example, the "stuff" you run in a browser will be a complete application as opposed to individual UI components, high-latency situations are problematic, and you cannot run application code on the client side. (I hope the RAP team will let me know in the comments if I got any of this wrong.)

The work on SWT BE "browser edition" is another take at the same problem, taking, if you will, the opposite approach: Client code (application and UI logic) makes use of the SWT API and as such can run in a desktop situation just fine. For "making it run in a browser", all the code is cross-compiled - to JavaScript for the Dojo port of SWT, or to ActionScript for the Flex port of SWT. So in a nutshell, instead of placing all application code on the server as with RAP, all application code ends up running on the client, in a browser. The obvious problem with this approach is that you still need code that runs on the server, to at least send data to the client and to receive updated data from the client, and that more often than not, you will end up with too much code on the client for execution. Of course, you could rewrite your existing code and ensure a proper client-server split, but if you are rewriting it anyway, why not go all the way and rewrite it using HTML and JavaScript?

For the e4 0.9 release, we have done the first step of exactly that - we rewrote or "cloned" an existing component using HTML, CSS, JavaScript, and Dojo on the client side. The idea is to gradually move certain components (not complete applications) over to use web technology, while still having these components appear in the desktop Eclipse client as first-class citizens. As a first example, we chose the PDE site.xml editor, because of its relative simplicity but also because it is a good representative of a whole class of form-based UI components in Eclipse. We wanted the end result to run in a regular browser, as well as an editor inside of Eclipse. Here you can see it running stand alone in a browser:

To make the editor work in a regular browser, it had to have a "real" server-side counterpart. Luckily, Eclipse already ships with an embedded servlet container, Jetty. This means that the web UI editor could talk to a servlet running in the same VM as the embedded browser widget rendering the UI. This is not unheard of at all - the Eclipse help system is taking the exact same approach.

However, just presenting the unchanged web UI in an embedded SWT browser widget - backed by IE if you run Windows, Firefox on Linux, and Safari on the Mac - is not enough. The editor would only be a first-class citizen in the Eclipse UI if, for example:
  • you didn't have to authenticate to the web UI every time you open an editor,
  • the editor signaled unsaved changes through the usual "*" in the editor tab, as opposed to an indicator in the editor itself,
  • you can trigger saving your changes using the menu (File>Save ),
  • etc.
In other words, the editor needs to make use of appropriate Eclipse APIs that connect Eclipse UI components such as views and editors to their context: the Eclipse workbench. One of the goals of e4 has always been to simplify the core APIs, and to make them accessible from other programming languages. So in a sense, we have started to make this real by offering JavaScript APIs that can be called by web UI components: to signal that they have unsaved changes, to provide a callback that will save those changes, etc.

Give it a try! After downloading the e4 0.9 SDK, start it up with an empty workspace and create an update site project (Ctrl+3, 'nusp'). Add some features and categories to it using the regular PDE site.xml editor, save, and then right-click on the site.xml file and use "Open With" to open it with the web-based editor. It will look like this:

Please leave comments if you have any questions. I will try to write up more about this in the next couple of days, going into more technical detail.


André said...

awesome! I specially loved the architecutral analysis (client-side and server-side logic, etc.). I peronally believe that we'll have to move towards the web and embrace it. The power and speed google and mozilla invest to move apps to the web won't be easy to follow though. Where will start the desktop in a few years? Will there be any separation left? etc. etc.

Anonymous said...

Hi Boris,

instead of commenting on the details I would like to step back and comment on the big picture.

Speaking purely from my personal POV, I'm not yet convinced that e4 is going to be a compelling general purpose web framework (perhaps a compelling tooling framework with web capabilities).

Let's say we just graduated from university and want to build the next facebook. Would we rather learn framework XYZ or OSGi+Extension Points+SWT/JFace+Workbench Model+EMF+declarative UIs?

I think the sweet spot lies somewhere beyond of what the current 'js widget toolkits' offer but with drastically less functionality/complexity than e3/e4 do require. However I don't think we can get there without leaving our legacy behind.

Looking forward to a constructive discussion,


André said...

Hi Elias

I completely agree that e4's too complex to be attractive as plain web framework. I hardly can imagine that we want to compete with rails. Eclipse will probably always be a niche product where a user programmer has to develop a simple web based solution (display, edit, store data). Eclipse always had a steep learning curve and its main strength always was in enterprise scale apps. You dont need OSGI modularization for small rich clients. But I agree in general, we most likely have to improve in this area.
But that's just one side of the medal: how to attract adopters. On the other hand, we might win the race where you need a framework for a large scale web application. A case study (you posted) on RAP showed that a customer choose RAP instead of GWT because of the clean programming model and the presence of advanced widgets, compared to GWT. That might be the competition that e4's capable to win: Is e4 compelling where you're up to implement an app (not just a page).
Any thouts?

YouCanPlayBetter said...

"A case study (you posted) on RAP showed that a customer choose RAP instead of GWT because of the clean programming model and the presence of advanced widgets, compared to GWT."

@André: That would be me :-) But my company is just a regular technology user. My point was that RAP is a fully developed framework. The widgets themselves (that would be RWT) are no more advanced than GWT's, but GWT does not really do more (OK, besides RPC calls). So developing with GWT feels mostly like working with SWT/RWT.

Ed Bartsch said...

Hi Boris,

I would also like to comment on the big picture.

I agree that client technologies and Web technologies will converge. I also like the approach to let Web-based components run within Eclipse and communicate with the rest of Eclipse effectively making Eclipse a better browser. I agree that client-server based approaches like RAP are limited to certain scenarios and are not able to bring every highly-interactive client-based application into the Web.

On the other side, there is a lot of power (developer experience, exiting code and frameworks) with Java/Scala/OSGi/SWT/EMF/Workbench/etc that can not be cross-complied into AJAX/JavaScript/Flex and Co but is compelling to be used in Web-based or client-server scenarios.

At the end, a user doesn't care whether it is JavaScript or OSGi. What matters for a Web user is whether it can be a) automatically installed/updated, b) triggered/opened by a URL, c) run without local footprint (i.e. all user data on server, roaming etc), d) integrated with other Web technologies/frameworks like AJAX, mashups etc.

What if Eclipse would create a technology (similar to Java WebStart or other Web browser plugins) that allows automatic installation/update/removal of "browser-embedded" Eclipse on the client machine? There can be also a JavaScript bridge that allows communication between "embedded" Eclipse and AJAX/JavaScript. Such an Eclipse installation must be able to run without local footprint (with exception of the p2 repository with OSGi bundles and necessary UI native libs) and communicate with some (eventually Eclipse-based) server to get/put user data and store user context.

The proposal goes in the same direction as RAP with regard to a client-server approach and single-sourcing. But the communication with backend parts of the application will be done not on the UI level as with RAP (where you need a high-frequency, low-latency channel) but on data/content access level where you can work with a low-frequency communication and may also use caching frameworks like Google Gears or CDO.

Such a solution is not intended to compete with Web frameworks where there is no chance for Eclipse to win. It will be rather a competition with Adobe AIR and JavaFX. I believe that Eclipse has very good chances to become a dominant technology in this space.

What do you think?

Best regards,

Sam said...

I don't want to comments on the complexity of e4 architecture. With the embedding web UI components in the Eclipse, it at least give the advantage of reusing the web widgets in Eclipse.
So question here - whether e4 can allow the embedded web UI components access data in local file system