I have been recently working on Pixel Perfect extension that allows web designers to overlay a page with semi transparent image and tweak the page HTML/CSS with per pixel precision - till it's matching the overlay.
This extension hasn't been working for several years (not maintained) and since requested by many users Firebug Working Group (FWG) got the opportunity to build that again and on top of native Developer tools in Firefox.
We had two goals in mind when building the extension:
- Make the Pixel Perfect feature available again
- Show how to build a real world extension on top of native API and tools in Firefox
This post focuses on the internal architecture. There is another post if you rather interested in the feature itself.
The extension is based on Add-on SDK as well as native platform API and we are also using JPM command line tool for building the final XPI.
There are several design decisions we made:
- Support upcoming multiprocess browser (e10s)
- Support remote devices (connected over RDP)
- Use known web technologies to build the UI (ReactJS)
These decisions had obviously an impact on the internal architecture described below.
Multiprocess Browser (e10s)
There are already articles about upcoming multiprocess support in Firefox, so just briefly what this means for extensions => specifically for Pixel Perfect. The basic stone in this concept is that web page is running in different process than the rest of the browser (where rest of the browser includes also extensions).
- Pixel Perfect 2 UI (the rest of the browser) is on the left side. It can't access the page content directly since it runs in different process (or it can even run on remote device, but more about that later).
- Web Page is on the right side, it runs in its own content process. It's secure (one of the main points of e10s) since it isn't that simple to access it (not even for extension developers
Pixel Perfect 2 (PP2), needs to properly cross the process boundary, setup messaging and deliver an image layer into the page content. The communication between processes is done using message managers that usually exchange JSON based packets with data.
Remote Debugging Protocol
Another challenge is making new features remotable. Don't worry, there is already API in place that allows implementing astounding things.
The user scenario for PP2 is as follows:
- The user runs Developer Toolbox and PP2 on his desktop machine.
- The Toolbox connects to an instance of Firefox running on a remote device.
- The user picks a new layer (an image file) on his desktop.
- The layer (image) appears inside loaded web page on the remote device.
Awesome, right? The user can tweak even a mobile device screens to pixel perfection!
PP2 needs to figure out yet a bit more to be remotable. There is not only a process boundary, but also a network boundary to cross. Let's see new and more detailed picture.
- Pixel Perfect 2 UI runs on the client side (in the chrome process). This is the desktop machine in our use case. It communicates with a Front object (RDP terminology).
- Front represents a proxy to the content process (local or remote). Executing methods on this object causes sending RDP packet cross processes and the network to the back-end (locale or remote device). All through RDP connection.
- Actor is an object that receives packets from the Front object. The Actor already lives in a content process (in case of PP2), and so it has direct access to the web page. This object is responsible for rendering layers (images) within the page. Actor also sends messages back (e.g. when layers are dragged inside the page, to update coordinates on the client).
- In-page Layer This is the image rendered over the page content. Note that images are not inserted directly into the page content DOM (that could be dangerous). They are rather rendered within a canvas that overlays the entire page. There are platform API for this and element highlighter (used by the Inspector panel) is also using this approach.
One thing to note, the RDP connection between Front and Actor crosses the network boundary as well as gets into a content process on the back-end automatically. It's also possible to create an Actor that lives in the chrome process on the back-end. But more about that in another post (let me know if you are interested).
Pixel Perfect 2 UI consists from one floating popup window that allows layer registration. The window looks like as follows.
Btw. the popup can be opened by clicking on a button available in the main Firefox toolbar (and there is also a context menu with links to some online resources).
Implementing user interface is often hard and one of our goals was also showing how to use well known web technologies when building add-on. The popup window consists from one
<iframe> element that loads standard HTML page bundled within the add-on package (xpi). The page is using RequireJS + ReactJS web stack to build the UI. Of course you can use any library you like to generate the markup.
There is yet another great thing. The frame is using content privileges only (
resource:// protocol for the content URL), not chrome privileges at all. It's safe just like any page loaded from the wild internet.
Let's sum everything up and see the final picture linked with the actual source code.
The explanation goes from top to bottom (client side -> back-end) starting with the PP2 UI.
- popup.html This is the Popup window. It consists from bunch of ReactJS templates (see the files in the same directory). The communication with
PixelPerfectPopupobject (pixel-perfect-popup.js) that lives in chrome content is done through message manager and JSON packets. Everything what lives in the data directory has content privileges. Stuff in lib directory has chrome privileges.
PixelPerfectPopupobject is implemented in this module. It's responsible for communication with the popup window as well as communication with the back-end through
PixelPerfectFront(pixel-perfect-front.js). If the user appends a new layer
panel.htmlsends a new event to
PixelPerfectPopup. It stores the layer in local store (a json file) and sends packet including an image data to the back-end
PixelPerfectActor. The Actor gets access to the Canvas and renders the image.
PixelPerfectStoreis implemented in this file. It's responsible for layer persistence. All is stored inside a JSON file within the current browser profile directory.
PixelPerfectFrontis implemented in this file. It represents the proxy to the backend. The code is nice and simple, most of the stuff is handled by the RDP protocol automatically.
PixelPerfectActoris implemented in this file. This file (a module) is loaded and evaluated on the backend. So, carefully with module dependencies, the backend can be a mobile device. All necessary stuff need to be sent from the client (e.g. a stylesheet). The actor uses Anonymous Content API and renders the layer/image received from the client. It also sends events back to the client. E.g. if a layer is dragged within the page, it sends new coordinates to the
PixelPerfectFront, then it's forwarded to
PixelPerfectPopupand further to
popup.htmlto update the final ReactJS template.
If you are ReactJS fan, you'll love the code. The JSON packet received all the way from the back-end actor (crossing process, network and security boundaries) is finally passed to
panel.setState(packet)method to automatically update the UI. Oh, yeah, pure pleasure for passionate developer
We (Firebug Working Group) care a lot about extensibility of native developer tools in Firefox and as we are making progress on new generation of Firebug we are also building new extensible API on the platform. If you want to know more about how to build developer (or designer) tool extensions, stay tuned. The next post will start a fresh new tutorial: Extending Firefox Developer Tools
Jan 'Honza' Odvarko