Subscribe to RSS Feed

I have been recently working on Pixel Perfect extension that allows web designers to overlay a page with semi transparent image and tweak the page HTML/CSS with per pixel precision - till it's matching the overlay.

This extension hasn't been working for several years (not maintained) and since requested by many users Firebug Working Group (FWG) got the opportunity to build that again and on top of native Developer tools in Firefox.

We had two goals in mind when building the extension:

  • Make the Pixel Perfect feature available again
  • Show how to build a real world extension on top of native API and tools in Firefox

This post focuses on the internal architecture. There is another post if you rather interested in the feature itself.

Requirements

The extension is based on Add-on SDK as well as native platform API and we are also using JPM command line tool for building the final XPI.

There are several design decisions we made:

  • Support upcoming multiprocess browser (e10s)
  • Support remote devices (connected over RDP)
  • Use known web technologies to build the UI (ReactJS)

These decisions had obviously an impact on the internal architecture described below.

Multiprocess Browser (e10s)

There are already articles about upcoming multiprocess support in Firefox, so just briefly what this means for extensions => specifically for Pixel Perfect. The basic stone in this concept is that web page is running in different process than the rest of the browser (where rest of the browser includes also extensions).

  • Pixel Perfect 2 UI (the rest of the browser) is on the left side. It can't access the page content directly since it runs in different process (or it can even run on remote device, but more about that later).
  • Web Page is on the right side, it runs in its own content process. It's secure (one of the main points of e10s) since it isn't that simple to access it (not even for extension developers 😉

Pixel Perfect 2 (PP2), needs to properly cross the process boundary, setup messaging and deliver an image layer into the page content. The communication between processes is done using message managers that usually exchange JSON based packets with data.

Remote Debugging Protocol

Another challenge is making new features remotable. Don't worry, there is already API in place that allows implementing astounding things.

The user scenario for PP2 is as follows:

  • The user runs Developer Toolbox and PP2 on his desktop machine.
  • The Toolbox connects to an instance of Firefox running on a remote device.
  • The user picks a new layer (an image file) on his desktop.
  • The layer (image) appears inside loaded web page on the remote device.

Awesome, right? The user can tweak even a mobile device screens to pixel perfection!

PP2 needs to figure out yet a bit more to be remotable. There is not only a process boundary, but also a network boundary to cross. Let's see new and more detailed picture.

  • Pixel Perfect 2 UI runs on the client side (in the chrome process). This is the desktop machine in our use case. It communicates with a Front object (RDP terminology).
  • Front represents a proxy to the content process (local or remote). Executing methods on this object causes sending RDP packet cross processes and the network to the back-end (locale or remote device). All through RDP connection.
  • Actor is an object that receives packets from the Front object. The Actor already lives in a content process (in case of PP2), and so it has direct access to the web page. This object is responsible for rendering layers (images) within the page. Actor also sends messages back (e.g. when layers are dragged inside the page, to update coordinates on the client).
  • In-page Layer This is the image rendered over the page content. Note that images are not inserted directly into the page content DOM (that could be dangerous). They are rather rendered within a canvas that overlays the entire page. There are platform API for this and element highlighter (used by the Inspector panel) is also using this approach.

One thing to note, the RDP connection between Front and Actor crosses the network boundary as well as gets into a content process on the back-end automatically. It's also possible to create an Actor that lives in the chrome process on the back-end. But more about that in another post (let me know if you are interested).

User Interface

Pixel Perfect 2 UI consists from one floating popup window that allows layer registration. The window looks like as follows.

Btw. the popup can be opened by clicking on a button available in the main Firefox toolbar (and there is also a context menu with links to some online resources).

Implementing user interface is often hard and one of our goals was also showing how to use well known web technologies when building add-on. The popup window consists from one <iframe> element that loads standard HTML page bundled within the add-on package (xpi). The page is using RequireJS + ReactJS web stack to build the UI. Of course you can use any library you like to generate the markup.

There is yet another great thing. The frame is using content privileges only (type="content" and resource:// protocol for the content URL), not chrome privileges at all. It's safe just like any page loaded from the wild internet.

Architecture

Let's sum everything up and see the final picture linked with the actual source code.

The explanation goes from top to bottom (client side -> back-end) starting with the PP2 UI.

  • popup.html This is the Popup window. It consists from bunch of ReactJS templates (see the files in the same directory). The communication with PixelPerfectPopup object (pixel-perfect-popup.js) that lives in chrome content is done through message manager and JSON packets. Everything what lives in the data directory has content privileges. Stuff in lib directory has chrome privileges.
  • pixel-perfect-popup.js PixelPerfectPopup object is implemented in this module. It's responsible for communication with the popup window as well as communication with the back-end through PixelPerfectFront (pixel-perfect-front.js). If the user appends a new layer panel.html sends a new event to PixelPerfectPopup. It stores the layer in local store (a json file) and sends packet including an image data to the back-end PixelPerfectActor. The Actor gets access to the Canvas and renders the image.
  • pixel-perfect-store.js PixelPerfectStore is implemented in this file. It's responsible for layer persistence. All is stored inside a JSON file within the current browser profile directory.
  • pixel-perfect-front.js PixelPerfectFront is implemented in this file. It represents the proxy to the backend. The code is nice and simple, most of the stuff is handled by the RDP protocol automatically.
  • pixel-perfect-actor.js PixelPerfectActor is implemented in this file. This file (a module) is loaded and evaluated on the backend. So, carefully with module dependencies, the backend can be a mobile device. All necessary stuff need to be sent from the client (e.g. a stylesheet). The actor uses Anonymous Content API and renders the layer/image received from the client. It also sends events back to the client. E.g. if a layer is dragged within the page, it sends new coordinates to the PixelPerfectFront, then it's forwarded to PixelPerfectPopup and further to popup.htmlto update the final ReactJS template.
    If you are ReactJS fan, you'll love the code. The JSON packet received all the way from the back-end actor (crossing process, network and security boundaries) is finally passed to panel.setState(packet) method to automatically update the UI. Oh, yeah, pure pleasure for passionate developer 😉

That's it for now. The rest is in the source code (there are a lot of comments, ping me if you need some more).

 

We (Firebug Working Group) care a lot about extensibility of native developer tools in Firefox and as we are making progress on new generation of Firebug we are also building new extensible API on the platform. If you want to know more about how to build developer (or designer) tool extensions, stay tuned. The next post will start a fresh new tutorial: Extending Firefox Developer Tools

Resources

Jan 'Honza' Odvarko


Rss Commenti

3 Comments

  1. [...] This post is about the feature, but if you are an extension developer interested in learning how to build developer or designer tools using the latest Firefox APIs, you might want to read about its internal architecture. [...]

    #1 Pixel Perfect 2: extension for Firefox Developer Tools | SDK News
  2. [...] This post is about the feature, but if you are an extension developer interested in learning how to build developer or designer tools using the latest Firefox APIs, you might want to read about its internal architecture. [...]

    #2 Pixel Perfect 2: extension for Firefox Developer Tools - InfoLogs
  3. [...] Pixel Perfect 2, Developer Tool Extension Architecture – Some notes on building a developer tool in the current versions of Firefox. [...]

    #3 Double Shot #1499 | CodangoNet

Sorry, the comment form is closed at this time.