Comments on: Creating a sample bugzilla database using kettle http://blog.mozilla.com/data/2009/06/05/creating-a-sample-bugzilla-database-using-kettle/ Mozilla metrics team's technical articles Fri, 28 Aug 2009 18:54:26 -0700 http://wordpress.org/?v=2.8.6 hourly 1 By: Nicholas Goodman http://blog.mozilla.com/data/2009/06/05/creating-a-sample-bugzilla-database-using-kettle/comment-page-1/#comment-9 Nicholas Goodman Thu, 11 Jun 2009 15:36:14 +0000 http://blog.mozilla.com/data/?p=13#comment-9 Daniel is spot on.... The current SQR does a horribily inefficient processing on the bug changes. Making it smaller to do the development (to simply change the ETL to use Analytic Query step) makes perfect sense. Adding back in the 5 million records should certainly be feasible, once the ETL is refactored. Daniel is spot on…. The current SQR does a horribily inefficient processing on the bug changes. Making it smaller to do the development (to simply change the ETL to use Analytic Query step) makes perfect sense. Adding back in the 5 million records should certainly be feasible, once the ETL is refactored.

]]> By: deinspanjer http://blog.mozilla.com/data/2009/06/05/creating-a-sample-bugzilla-database-using-kettle/comment-page-1/#comment-4 deinspanjer Mon, 08 Jun 2009 12:28:34 +0000 http://blog.mozilla.com/data/?p=13#comment-4 The bmo bugs_activity table has several million records in it. The size of this table was a limitation due to the way the ETL is processing history currently. The only tables we did special processing on were bugs, bugs_activity, and users. The other tables that we didn't copy were left out simply to make it easier to develop, test, and debug out ETL processes. The bmo bugs_activity table has several million records in it. The size of this table was a limitation due to the way the ETL is processing history currently. The only tables we did special processing on were bugs, bugs_activity, and users.

The other tables that we didn’t copy were left out simply to make it easier to develop, test, and debug out ETL processes.

]]> By: Max Kanat-Alexander http://blog.mozilla.com/data/2009/06/05/creating-a-sample-bugzilla-database-using-kettle/comment-page-1/#comment-3 Max Kanat-Alexander Mon, 08 Jun 2009 04:42:26 +0000 http://blog.mozilla.com/data/?p=13#comment-3 This is really incredible and involved, but for the copying part, why didn't you just copy everything except for longdescs and attach_data? Those are the only two tables that are of any size. All of the other tables are pretty tiny, relatively. -Max This is really incredible and involved, but for the copying part, why didn’t you just copy everything except for longdescs and attach_data? Those are the only two tables that are of any size. All of the other tables are pretty tiny, relatively.

-Max

]]>