Log in

No account? Create an account
on metrics... - max spevack's blog [entries|archive|friends|userinfo]

[ website | My Website ]
[ userinfo | livejournal profile ]
[ archive | journal archive ]

on metrics... [Mar. 7th, 2007|11:07 am]
[Location |raleigh, nc]

So this is the link that I am always pointed to, whenever there is any sort of a conversation about "number of users" with Fedora. It's an interview that Mark Shuttleworth gave to Red Herring, dated December 29th, 2006.



Q: What about growth in adoption rates, any kind of numbers that you can give me?
A: We know now that there are probably at least 8 million [Ubuntu] users.


And then they move on to another question.

So here's my question for anyone who is In The Know. Where does that number come from? Can you please give us some context?

And here's my question for the interviewer. How can you not follow up that question by asking for a bit of context? Seriously? You're going to let someone get away with an answer as heavily hedged as "probably at least" and then not dig a bit deeper?

I don't necessarily enjoy tracking all of the Fedora usage, but it is a necessary evil. We spend tons of time, energy, and money developing Fedora, and it's our responsibility to do the best we can to understand what the fruits of that labor are. I'll also be the first to say that with things like user statistics, you can never get it 100% right, all you can do is try to reduce the unknowns as much as possible and be honest about the grey areas.

And I believe that we do an honest, transparent job of this in Fedora. And it really bothers me when people look at our numbers, and look at the places where we acknowledge the potential flaws in our methods, and then flame us. Not everyone does, but enough people do to actually get me to write this blog post.

I ask myself -- why would people rather see a claim like "8 million Ubuntu users" with ZERO explanation, than our Fedora statistics that actually try to talk about where this stuff came from?

In the OSS world, which prides itself on transparency, I find this baffling.

[User Picture]From: jspaleta
2007-03-07 06:02 pm (UTC)

Going Beyond Metrics.... Statistics

There are three kinds of lies: lies, damned lies, and statistics.--Mark Twain

So now that we have a generally useful methodology to collect numbers that are indicative to number of installed systems... can we datamine that in more arcane ways? I'd be interested for example to see what the reliable hourly,daily,weekly trends are in terms of client connections looking for updates. I'd like to think we could use that sort of information when planning disruptive infrastructure changes that affect the download to minimize the impact on users.

(Reply) (Thread)
[User Picture]From: spot
2007-03-08 03:19 am (UTC)
They mailed out 8 million CDs, that must be 8 million users, right? :)
(Reply) (Thread)
From: (Anonymous)
2007-05-04 07:55 am (UTC)

It might be an extrapolation from popcon

See http://popcon.ubuntu.com/ . However I think its more likely it was based on downloads or people fetching updates...
(Reply) (Thread)