DR. HANDCOCK: Leading into the next session, could you make some comments on the reliability and quality issues and, in particular, about missing sample data?
DR. CARLEY: Well, the first thing I will say is that these metrics in here don't yet have confidence intervals about them, showing how confident we are that this metric is what it is given the high levels of missing data. So, that is an unsolved problem. The second thing is that we have been doing a series of analyses, as I know several other people here have been doing similar analyses, looking at networks where we know what the true answer is, and we then sample from it, and we then re-estimate the measures to see how robust they are.
Steve Borgatti and I have just finished some work in that area, which suggests that, for a lot of these measures, if you have 10 or 20 percent errors, that as long as you are not trying to say, this person is number one, but instead say, this person is in the top 10 percent, you are going to be about right some 80-90 percent of the time. So, it has that kind of fidelity. Is it worse or better with particular types of network structures? We are not sure yet. Some of our leading data suggests that for some types of network structures, like corporate free networks, you can in fact do better than that.