Content of and Plan for ‘Meta-Analysis Sandwich’

Welcome to my first ever blog.  In this inaugural post I’ll comment on topics I anticipate covering, a few challenges of blogging about statistics, and my primary reasons for undertaking this endeavor.

Anticipated Topics

This blog is about meta-analysis methodology and related topics.  Let’s break that down:

• I usually use “meta-analysis” to refer to statistical techniques for combining and comparing quantitative results across similar studies.  Several aspects of this loose definition are worth discussing (e.g., what’s a study? how similar? results in what form?), but not in this post.  Some authors’ terms such as “pooled analysis” or “integrative data analysis” fall under what I consider meta-analysis.
• By “methodology” I mean the how-to of meta-analysis: I’ll focus on procedures and concepts that meta-analysts use to conduct meta-analyses, as well as attendant choices, challenges, and other issues involved in responsible meta-analytic practice.  I’ll address specific substantive applications of meta-analysis only rarely, and then mainly to illustrate methodological points.
• Related topics include other methodological aspects of the research syntheses—overviews, systematic reviews, evidence-based reviews, health technology assessments, etc.—in which meta-analyses are often embedded.  These topics pertain to numerous activities that meta-analysis tends to follow (e.g., conducting, reporting, finding, evaluating, and coding primary studies) or precede (e.g., reporting and updating reviews; using reviews to inform primary research, policy, or practice).

So, the key ingredient of my ‘Meta-Analysis Sandwich’ blog is meta-analysis, but we’ll also consider the “fixings” and slices of bread.  (Hmmm … would ‘Meta-Analysis Hotdish’ be more apt?)  We’ll have abundant food for thought: The number of articles and other work on relevant methodology published or otherwise disseminated in one recent year (2009) alone is between about 500 and 2,500—depending heavily on what’s considered relevant.

I hope to make approximately one post per week.  Its topic will often be rather narrow but widely applicable, focused on selected aspects of a specific issue that many meta-analysts encounter.  That said, I’ll sometimes address topics that are fairly diffuse (e.g., bibliographies on research synthesis) or of interest to a small fraction of meta-analysts (e.g., meta-analytic path analysis).  As a hint at topics I’m likely to cover, here’s a casual list of ideas, most of which could easily give rise to at least half a dozen posts:

• various types of effect size (ES)
• extracting ES estimates in challenging situations
• conditional/sampling variances for ES estimators
• coding and analyzing study features
• between-studies/interstudy heterogeneity
• fixed-effect(s) vs. random-/mixed-effects models
• study-level moderators/covariates
• graphical displays of ESs and meta-analytic results
• missing or degraded data
• outliers or other aberrant cases
• (non)publication bias and related biases
• multivariate ESs and other dependent ESs
• Bayesian and empirical Bayes approaches
• bootstrapping and other resampling techniques
• Monte Carlo simulations
• resources (e.g., training, software, bibliographies)

Depending on my interests and feedback from readers, I might also address specialized or rare methods such as reliability or validity generalization, meta-analytic structural equation modeling, network meta-analysis (i.e., mixed-treatment comparisons), meta-analysis of diagnostic test accuracy, cumulative meta-analysis, meta-analysis of individual patient/participant data, combining p values, and vote counting.  Whatever the topic, I’ll try to cite relevant work amply.

Blogging about Statistics

Writing about statistical methods to a diverse audience (of mostly strangers) can be tricky.  I plan to gear most of my posts toward readers who are statistically literate but aren’t statisticians by training; I suspect this describes many practicing meta-analysts and interested others.  Where feasible I’ll illustrate focal concepts or procedures using real or fictitious data.  I’ll usually avoid excessive technical detail and mathematical rigor, though “excessive” is subjective.  When writing equations and other mathematical expressions, I’ll try to use notation that’s precise without being too cumbersome.  Please be patient as I learn how to convey mathematics in a blog environment.

I anticipate that some posts will warrant supplemental material.  For instance, a useful presentation on certain topics will benefit from technical info (e.g., derivation, procedural details), computing tools (e.g., code snippets, executable programs, spreadsheets), or extended examples that are too long or complicated for a typical blog entry.  How best to provide this material is unclear—external web page? download? email request?—but my strategies should improve with experience.

Motivating Factors

Finally, for anyone still reading I’ll mention the top reasons I decided to write this blog despite having scant spare time.  Some are altruistic; others, blatantly self-serving.

• Outreach: Meta-analysis has been my main research area since 1997, and I’d like to share some of what I’ve learned to promote responsible practice—especially among conscientious, diligent meta-analysts who don’t keep up with the diverse, large, and rapidly growing methodological literature in this area.
• Advertising: I earn my living as an independent statistical consultant, paid by clients to help design studies, analyze data, and disseminate findings; this blog will serve as an extensive free sample to acquaint prospective clients with my knowledge and resources in a specific domain as well as my styles of thinking, writing, and interacting.
• Drafts: Some of my posts will be refined into workshop presentations and other products or services I can offer to generate consulting income.
• Feedback: Similarly, I might eventually publish some posts about novel methods in peer-reviewed outlets; constructive criticism on these ideas from methodologists and applied researchers will help improve the methods and exposition thereof.

That’s all for now.  I look forward to sharing ideas about the wide world of meta-analysis.