Melissa G Wolf, PhD | Melissa G Wolf

RESEARCH SKILLS

Surveys, Psychometrics, Research Methods, Linear and Logistic Regression, Mixed Effects Regression, Multivariate Regression, Structural Equation Modeling, Exploratory Factor Analysis, Confirmatory Factory Analysis, Item Response Theory, Cognitive & User Interviews, Focus Groups, Causal Inference, Network Analysis, Mixture Modeling, Ethnography, AB Testing, Machine Learning, Growth Models, Moderated/Unmoderated Usability Testing, Cluster Analysis, MaxDiff, ANOVA, Concept Testing, Web scraping, Text mining, Sentiment Analysis, Benchmarking, Data Visualization

TECHNOLOGY

R, Shiny, SPSS, Mplus, Qualtrics, Google Analytics, Python, SQL, KQL, Stata, HLM, GPower, GitHub, MTurk, Excel, PowerBI, JavaScript, HTML, CSS, Survey Monkey, Figma, UserTesting, Selenium

WORK

Statistical Programmer at Delfi Diagnostics (Feb 2023 - Present)
User Experience Researcher at Microsoft - Contract (Feb 2022 - Feb 2023)
- Derive insights from quantitative and qualitative data, complementing research performed autonomously and in partnership with colleagues (quantitative analyses performed using R, Power BI, and KQL).
- Plan and conduct 10+ end-to-end research projects using generative and evaluative designs, producing actionable next steps for design and engineering to improve products for users.
- Perform moderated and unmoderated usability tests to guide product development, contributing to 30% increase in MAU and $27M in revenue.
- Spearhead a special project to develop and validate a comprehensive survey to measure several dimensions of user experience for Microsoft Web products, a KPI for multiple products.
- Debug and overhaul of a companywide UX testing platform, updating the statistical methods and approaches used in AB testing to compare UI designs.
- Incorporate telemetry data into research projects to supplement findings from user interviews in order to enhance understanding of all dimensions of the user experience.
- Synthesize analyses into actionable reports for designers, PMs, and developers, and presented findings in division wide meetings.
Statistical Programming Consultant at Arizona State University (Mar 2020 - Present)
- Own the end-to-end development and production of a Shiny App and R package to derive model specific cutoffs for factor models, modernizing measurement of psychological constructs and validation of surveys.
- Build data visualizations using R and ggplot2 to facilitate easy comprehension of results for users
- Set up Google Analytics dashboards to identify most popular products to guide future development and maximize DAU
- Achieved a substantial reduction in of monthly bug reports (from five to zero) through implementation of error messages that identify the most likely solution to the user.
- Received recognition with the award of a highly competitive $550,000 federal grant in 2022; continue to provide consultancy services following receipt of the grant
Data Consultant at New Tech Network (Jul 2021 - Jun 2022)
- Attained an 80% increase in time savings though automating weekly reports with 1000’s of lines of data (using R and Selenium), resulting in an extension to consultancy contract.
- Held accountability for issuing surveys, compiling survey data, analyzing results, publishing survey reports, and communicating with representatives from 49 schools.
Psychometric Consultant on John Templeton Foundation Grant #61187 (Sept 2017 - Jun 2021)
- I co-created an iterative mixed method approach to item construction and item-level validation called the Response Process Evaluation method. This method is like a cognitive interview, but more efficient, cost-effective, and easily scalable to hundreds of participants. I used this method to collect mixed methods data from over 3,000 participants in America and India to develop and validate a survey.
Graduate Student Researcher on IES Grant #R305A160157 (Sept 2016 - Jun 2019)
- I used factor analysis (Mplus) to test the internal structure of a scale and employed mixture models to evaluate if the construct would be better conceptualized as categorical.
- I helped collect and clean data, and worked on methods to detect deceptive and/or unusual response patterns.
Research Analyst at International Baccalaureate (Jun 2012 - Aug 2015)
- Own end-to-end development of 10+ research projects. Analyzed quantitative data from a large internal database to co-author white papers and journal articles (statistical analyses performed using SPSS, HLM, and Excel).
- Design QA surveys, distributing them to 1000+ people weekly. Prepare and present reports to internal stakeholders.
- Develop RFPs and evaluate submissions, providing extensive feedback for external collaborators.
- Attain a 60% increase in time savings through automation of data cleaning (using Excel macros and SPSS syntax) that decreased weekly survey preparation and launching time.

EDUCATION

Doctor of Philosophy (2022): Quantitative Methods for the Social Sciences
University of California, Santa Barbara
Committee: Andrew Maul, Karen Nylund-Gibson, Allison Horst, Ann Taves, Dan McNeish

Master of Arts (2020): Education
University of California, Santa Barbara

Master of Arts (2017): Research Methods and Statistics
University of Denver

Graduate Certificate (2012): Measurement, Statistics and Evaluation
University of Maryland, College Park
Advisor: Gregory R. Hancock

Bachelor of Arts (2009): Communication
University of Delaware

TEACHING EXPERIENCE

Teaching Assistant at University of California, Santa Barbara (2019 – 2021)
ED214A: Introduction to Statistics (2x)
ED214B: Inferential Statistics (2x)
ED214C: Linear Models
SOC108: Introduction to Research
UCSB-Smithsonian Scholars Program: Introduction to Data Science

Teaching Assistant at University of Maryland, College Park (2010 – 2012)
EDMS645: Quantitative Methods
EDMS610: Classroom Assessment

RESEARCH

Publications

Wolf, M. G. & McNeish, D. (2022). dynamic: An R Package for Deriving Dynamic Fit Index Cutoffs for Factor Analysis. Multivariate Behavioral Research.
McNeish, D. & Wolf, M. G. (2022). Dynamic fit index cutoffs for one-factor models. Behavior Research Methods.
Boness, C.L., Helle, A.C., Miller, M.B, Wolf, M.G., & Sher, K.J. (2022). Who opts in to alcohol feedback and how does that impact behavior? A pilot trial. Journal of Studies on Alcohol and Drugs, 83(5), 640-645.
Wolf, M. G., Ihm, E., Maul, A., & Taves, A. (2022). Survey item validation. In S. Engler & M. Stausberg (Eds.), Handbook of Research Methods in the Study of Religion (2nd ed.). Routledge.
McNeish, D. & Wolf, M. G. (2021). Dynamic Fit Index Cutoffs for Confirmatory Factor Analysis Models. Psychological Methods. https://doi.org/10.1037/met0000425
Clairmont, A., Wolf, M. G., & Maul, A. (2021). The prevention and detection of deception in self-report survey data. In U. Luhanga & G. Harbaugh (Eds.), Basic Elements of Survey Research in Education: Addressing the Problems Your Advisor Never Told You About. Charlotte, NC: Information Age Publishing.
McNeish, D., & Wolf, M.G. (2020). Thinking twice about sum scores. Behavior Research Methods. https://doi.org/10.3758/s13428-020-01398-0
Luo, Y. & Wolf, M. G. (2019). Item parameter recovery for the two parameter testlet model with different estimation methods. Psychological Test and Assessment Modeling, 61(1), 65-89.
Ghafoori, B., Wolf, M. G., Nylund-Gibson, K., & Felix, E. D. (2019). A naturalistic study exploring mental health outcomes following trauma-focused treatment among diverse survivors of crime and violence. Journal of Affective Disorders, 245, 617–625. https://doi.org/10.1016/j.jad.2018.11.060
Raines, T.C., Gordon, M., Harrell-Williams, L.M., Diliberto, R.A, & Parke, E.M. (2017). Adaptive skills and academic achievement in Latino students. Journal of Applied School Psychology, 245 - 260. https://doi.org/10.1080/15377903.2017.1292974
Gordon, M., VanderKamp, E. & Halic, O. (2015). Research brief: International Baccalaureate programmes in Title I schools in the United States: Accessibility, participation and university enrollment. https://www.ibo.org/globalassets/publications/ib-research/title-1-schools-research.pdf
Bergeron, L. & Gordon, M. (2015). Establishing a STEM pipeline: Trends in male and female enrollment and performance in higher level STEM courses. International Journal of Science and Mathematics Education, 1 - 18. http://dx.doi.org/10.1007/s10763-015-9693-7
Gordon, M., & Bergeron, L. (2014). The use of multilevel modeling and the level two residual file to explore the relationship between Middle Years Programme student performance and Diploma Programme student performance. Social Science Research, 50, 147-163. https://doi.org/10.1016/j.ssresearch.2014.11.004

Under Review

Wolf, M. G., Taves, A., Ihm, E. D., & Maul, A. (2023). The Response Process Evaluation Method. PsyArXiv. https://doi.org/10.31234/osf.io/rbd2x
Wolf, M. G. & Denison, A. J. (2023). Survey uses may influence survey responses. PsyArXiv. https://doi.org/10.31234/osf.io/c4hd6
Taves, A., Ihm, E., Wolf, M. G., Barlev, M., Kinsella, M., & Vyas, M. (2023). The Inventory of Nonordinary Experiences (INOE): Evidence of Validity in the United States and India. PsyArXiv. https://doi.org/10.31234/osf.io/r6bw9

Packages and Applications

Wolf, M. G. & McNeish, D. (2020). Dynamic Model Fit (version 0.1.0.). [Software]. Available from www.dynamicfit.app
Wolf, M. G. & McNeish, D. (2020). dynamic: Model fit cutoffs. R package version 0.1.0. https://cran.r-project.org/web/packages/dynamic/index.html

Select First Author Presentations

Wolf, M. G. & McNeish, D. (2022, April). Dynamic Model Fit Indices: A R Shiny Application. In D. Katz (Chair), Past, Present, and Future of Model Fit: Modern Solutions to Historical Challenges. Symposium conducted at the annual meeting of the American Educational Research Association.
Wolf, M. G., Ihm, E., Katz, D., & Maul, A. (2020, June). Improving Psychological Measurement: Introducing the RPE Method. Workshop presented at the annual meeting of the Society for the Improvement of Psychological Science, Virtual Meeting.
Wolf, M. G. (2020, April). The Response Process Evaluation Method. Paper presented at the meeting of the International Objective Measurement Workshop, San Francisco, CA.
Wolf, M. G. & Ihm, E. (2019, November). Validation Methods and Results of the Inventory of Non-Ordinary Experiences (INOE). In C. Kravette (Chair), Comparing Non-Ordinary Experiences across Cultures: Methodological Innovations and Findings from the US and India. Symposium conducted at the annual meeting of the American Academy of Religion.
Wolf, M. G., Clairmont, A., Maul, A., Furlong, M. J., & Stiblina, V. (2019, April). A Method for Detecting Invalid Responses. Paper presented at the annual meeting of the American Educational Research Association, Toronto, Canada.
Wolf, M. G., Nylund-Gibson, K., Dowdy, E., & Furlong, M. J. (2019, April). A comparison of categorical and continuous latent variable models in a moderation framework. In O. Simon (Chair), Mixture Modeling in Practice. Symposium conducted at the annual meeting of the American Educational Research Association, Toronto, Canada.
Wolf, M. G., & Bergeron, L. (2018, November). Using different data sources to address the same research questions: Evaluating the effectiveness of new curriculum on student outcomes. Paper presented at the annual meeting of the California Educational Research Association, Anaheim, CA.
Wolf, M. G. (2018, April). Validation in the absence of traditional constructs. Paper presented at the meeting of the International Objective Measurement Workshop, New York, NY.

Leadership

American Educational Research Association Division D Program Committee Graduate Student Representative, 2018 – 2019
Expert Advisory Board member at the Center for Mind and Culture, 2019 – Present
Course Director: Psychological Network Analysis, 2019
Research Methods and Statistics Student Association President, 2015 – 2016

Membership

Society for the Improvement of Psychological Science
American Educational Research Association
- Division D, Survey Research in Education SIG
National Council on Measurement in Education

Manuscript Reviewing

Interdisciplinary Journal of Problem-Based Learning
Meta-Psychology
Behavior Research Methods

Honors and Awards

Block Grant Dissertation Award, University of California, Santa Barbara (2020)
Department of Education Excellence Award for Research (2019)
Grad Slam Finalist (Top 9 out of 79) (2019)
Block Grant Fellowship Award, University of California, Santa Barbara (2018)
Education Travel Grant, University of California, Santa Barbara (2018–2019)
New Tech Network $10,000 Research Grant, Napa, CA (2016)
Block Grant Fellowship Award, University of California, Santa Barbara (2016)
University of Denver Graduate Student Travel Grant (2016)
University of Denver Scholarship Award (2015)
Dean’s Fellowship, University of Maryland, College Park (2010–2011)

Coursework

Structural Equation Modeling, Constructing Measures, Analyzing and Validating Measures, Item Response Theory, Psychological Network Analysis, Psychometrics, Bayesian Statistics, Mixture Modeling, Multi-Level Modeling, Causal Inference, Meta-Analysis, Empirical Research Methods, Program Evaluation, Applied Sampling, Survey and Design Analysis, Philosophy of Measurement, Introduction to SAS, Introduction to Simulation, Multivariate Data Analysis, Applied Measurement, Factor Analysis, Quantitative Research Methods I & II, Applied Multiple Regression Analysis, Classroom Assessment & Evaluation, Introduction to Qualitative Research, Ethnography, Anthropology of Education, Social Psychology