Recently Jorge Villalobos ran some tests to measure the impact of using the SDK on Firefox start-up time. Jorge published his finding to the Jetpack google news group. His results were encouraging:
The results were very favorable this time, with the
empty add-on adding only 1.8% (11ms) to startup, and Infocon adding only
3.6% (22ms), on average. As a reference point, after our initial tests
we decided that 7% was roughly the line where add-on startup overhead
stopped being “Excellent” and began being just “Good”.
The methodology of these tests was to measure Firefox start-up time under three different scenarios:
- using a clean profile with no addons enabled.
- with a single ‘empty’ SDK-based add-on enabled.
- with a non-trivial add-on enabled.
The great thing about this is that we now have solid data that indicates SDK version 1.3 itself has little effect on Firefox start-up time however, as Jorge suggests, there is much more testing that could be done. In particular, we should continue to run these tests to ensure that future versions of the SDK do not degrade start-up performance as we add new modules or make infrastructure changes to the SDK.
If you’re curious, we test start-up impact using Talos according to this methodology:
https://developer.mozilla.org/en/Measuring_Add-on_Startup_Performance
If you want to run the tests yourself, Talos is relatively straightforward to set up:
https://wiki.mozilla.org/StandaloneTalos#How_to_set_up_Talos_for_testing_at_home
I should warn you that the tests themselves can take quite some time to complete!
