A detailed look at UI performance testing using OrangeMark

OrangeMark is one of a handful of benchmarks that focus solely upon rendering performance. It’s comprehensive and covers 40 rendering techniques including some which are rarely used in modern UI design. Here we’ve drilled down to look at performance for the most commonly used subsets.

OrangeMark doubles the intensity of each test until the frame rate falls below 25 fps. A test gets a maximum score if the intensity has successfully been doubled 10 times, so a browser scoring 10 points is roughly twice as fast as one that scores 9 points.

The OrangeMark benchmark was developed in 2012 when 25 fps was considered a good performance target. Since then hardware and UI performance expectations have increased, so for our benchmarking we tweaked the frame rate target to 50 fps.

In order to focus on the most used UI rendering techniques, we looked at a wide number of HTML UIs from TV middleware products and OTT applications and used this information to group the 40 OrangeMark tests into 3 categories:

Important:
(12/40 tests)
Regularly used techniques such as scaling, linear gradients, text embossing, carousel rotation and canvas.
Infrequent:
(18/40 tests)
Less commonly used techniques such as 2D rotation, skewing, 3D animation and reflection.
Must Have:
(10/40 tests)
Essential properties, attributes and animations necessary to create stylish and intuitive UIs.

Our benchmarking was carried out on a MacBook Pro so we could compare against optimised releases of all the well-known browser engines and the most recent versions were used. The average score per category for each browser is recorded in the table below (higher is better).

FlowSafari
(WebKit)
Firefox
(Gecko)
Chrome
(Blink)
Must Have
9.709.008.109.40
Important8.507.926.508.25
Infrequent7.336.894.787.17

Average score for each category

As expected our new Flow browser outperforms the other browser engines in all categories. However, it’s hard to see how much faster because scoring just one point higher means the browser is twice as fast. Converting the performance scores onto a linear scale shows a clearer distinction between the different browsers.

FlowSafari
(WebKit)
Firefox
(Gecko)

Chrome
(Blink)
Must Have832512274676
Important36224291304
Infrequent16111927144

Linear scale performance (2x)

What does this mean for a content developer who is creating a new app?

The numbers speak for themselves. Even ignoring the infrequent category, Flow is 1.2 times faster than Chrome, 1.6 times faster than Safari and 3.3 times faster than Firefox. Choosing Flow as your browser engine will lead to a smoother and more responsive experience.

And for existing HTML middleware providers?

Because Flow supports HTML5, no content changes are needed to benefit from the improved performance. The categories we present here are based upon our analysis of a great many middlewares and apps and the data strongly supports switching your recommended browser to Flow.

What if I want to know more?

Further details on the Flow browser can be found in the Ekioh technical papers “Designing a browser to benefit from multi-core silicon” and  “An optimal browser design for GPU utilisation”. If you want more information on the specific OrangeMark tests that are in each category, or details on how to tweak the frame rate, please drop us an email.

More about Flow…


If you’d like to automatically receive our posts by email please join our mailing list.

BACK TO BLOG