Man Versus Machine? Self-Reports Versus Algorithmic Measurement of Publications / Xuan Jiang, Wan-Ying Chang, Bruce A. Weinberg.

By:

Jiang, Xuan

Contributor(s):

Material type: Text

TextSeries: Working Paper Series (National Bureau of Economic Research) ; no. w28431.Publication details: Cambridge, Mass. National Bureau of Economic Research 2021.Description: 1 online resource: illustrations (black and white)Subject(s):

Online resources:

Available additional physical forms:

Hardcopy version available to institutional subscribers

Abstract: This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare scientific publications collected by surveys and algorithmic approaches. We aim to illustrate the different types of measurement errors in self-reported and machine-generated data by estimating how publication measures from the two approaches are related to career outcomes (e.g. salaries, placements, and faculty rankings). We find that the potential biases in the self-reports are smaller relative to the algorithmic data. Moreover, the errors in the two approaches are quite intuitive: the measurement errors of the algorithmic data are mainly due to the accuracy of matching, which primarily depends on the frequency of names and the data that was available to make matches; while the noise in self reports is expected to increase over the career as researchers' publication records become more complex, harder to recall, and less immediately relevant for career progress. This paper provides methodological suggestion on evaluating the quality and advantages of two approaches to data construction. It also provides guidance on how to use the new linked data.

Tags from this library: No tags from this library for this title. Log in to add tags.

Average rating: 0.0 (0 votes)

Holdings
Item type	Home library	Collection	Call number	Status	Date due	Barcode	Item holds
Working Paper	Biblioteca Digital	Colección NBER	nber w28431 (Browse shelf(Opens below))	Not For Loan

Total holds: 0

Collection: Colección NBER Close shelf browser (Hides shelf browser)

Previous	No cover image available	No cover image available	No cover image available	No cover image available	No cover image available	No cover image available	No cover image available	Next
Previous	nber w28429 Data-Driven Incentive Alignment in Capitation Schemes /	nber w2843 Predicting Nursing Home Utilization Among the High-Risk Elderly /	nber w28430 Time Aggregation in Health Insurance Deductibles /	nber w28431 Man Versus Machine? Self-Reports Versus Algorithmic Measurement of Publications /	nber w28432 Is There A Replication Crisis In Finance? /	nber w28433 O-Ring Production Networks /	nber w28434 A Parsimonious Behavioral SEIR Model of the 2020 COVID Epidemic in the United States and the United Kingdom /	Next

February 2021.

This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare scientific publications collected by surveys and algorithmic approaches. We aim to illustrate the different types of measurement errors in self-reported and machine-generated data by estimating how publication measures from the two approaches are related to career outcomes (e.g. salaries, placements, and faculty rankings). We find that the potential biases in the self-reports are smaller relative to the algorithmic data. Moreover, the errors in the two approaches are quite intuitive: the measurement errors of the algorithmic data are mainly due to the accuracy of matching, which primarily depends on the frequency of names and the data that was available to make matches; while the noise in self reports is expected to increase over the career as researchers' publication records become more complex, harder to recall, and less immediately relevant for career progress. This paper provides methodological suggestion on evaluating the quality and advantages of two approaches to data construction. It also provides guidance on how to use the new linked data.