Interactions and main effects in simple linear models

Bet that title’s got you itching to read on! Feel free to skip this post if you think stats are boring. You’d be wrong, but I won’t judge you… Anyway, getting straight down to the nitty gritty, and assuming that if you’ve hung in until now then you’re not afraid of words like ‘linear’ and ‘model’, here’s the thing: when I learnt statistics, one thing that was drummed in to us was that, if you’re fitting a linear model which includes interactions, you can’t sensibly interpret the main effects. I’ve been telling people the same ever since. But, in almost every paper that I review or edit, and which uses such models, people do just that. My purpose here then is to explain why I think that’s wrong. And hopefully, to find out if I am wrong to think that way.

Let’s start with a contrived example. Suppose we’ve measured activity levels at different times of the day across 100 individual birds, 50 of which are larks and 50 are owls, and we get the following:

Of course, in analysing these data the obvious thing to do would be to fit a linear model with activity modelled as a function of hour, species, and their interaction. The fitted lines on the figure above illustrate this model. And the explanation is straightforward: activity increases towards dawn in larks, and decreases towards dawn in owls.

What doesn’t make sense is to make any statements about general differences between larks and owls, or about any general trend in activity with time from dawn, because these ‘main effects’ are completely entangled within their interaction.

Unfortunately, most statistical software packages will give an output which includes significance levels for both main effects and interactions, for instance:

Or in Anova form (R guys – I know this is wrong, but its particular flavour of wrongness is not important for this point!):

Both of these outputs make it look like the main effects are ‘significant’, and the coefficients even seem to tell you the direction of these effects. So the kind of interpretation I read again and again would look something like: “There is a significant interaction between activity and species (p < 0.0001). In addition, activity increases with hour from dawn (p < 0.0001), and is higher in owls than in larks (p < 0.0001).” Which, as we’ve just demonstrated, is nonsense. And just because this example has been contrived to emphasise this point, doesn’t mean it doesn’t apply equally whenever there is an interaction; sensible plotting of your data will usually reveal this.

What should you do instead? Report the significant interaction, and then describe it, using the table of coefficients (and your knowledge of how your stats package of choice uses aliasing) to calculate the slope (and intercept if you like) for each level in the interaction. Here for example, the slope for the relationship for larks is 1.04, and for owls is 1.00 (1.045 – 2.048) – and you can get confidence intervals for both easily enough. And if the interaction doesn’t seem to be important, take it out, and interpret main effects to your heart’s content.

That’s my take on it anyway. The very concept of the ‘significance’ of a main effects is meaningless in the presence of an interaction between them. And so I keep telling people. So if I’m missing something, please let me know!

The enduring pleasure of paper

I have tried very hard to go paper free. My printer is rarely turned on, almost all of my scientific reading and reviewing, editing and marking is now done on screen, and my iPad Papers library is full of PDFs destined to remain digital. Why, then, was it such a pleasure to return home yesterday to find, after a gap of a few months, a pristine, cellophane-wrapped copy of Nature on my doormat? Honestly, I really did try to kick this habit. Earlier this year my Nature subscription expired around the time that my blogging briefly reached the required frequency to qualify me for personal online access. I duly installed the Nature.com Reader on the iPad, marvelled as it filled with content, and… have barely looked at it since.

Partly, I suspect, this is due to my entrenched reading routines. Just as I need a physical copy of the Guardian every Saturday to satisfy my coffee and crossword habit, I had also established the 20 minute tram ride to work as the ideal environment in which to catch up on essential science news. Maybe down in that London or somewhere equally fancy it would be acceptable to flop out the iPad and carry on regardless. But I just can’t do that on a Sheffield tram without feeling what Logan Mountstuart would call, with spectacular rudeness, a CAUC (one for readers of William Boyd’s Any Human Heart).

But also, I know my way around the physical magazine – know where I can find the news, the books and arts section, the opinions, correspondence, news and views, all the bits I tend to read – and prefer its linear structure to the mass of interlinked stuff on the online (and App-ed) versions, which always leave me with the feeling that I may have missed something essential. Harder to ignore content when there is less choice too, I suspect.

All of which lends to the peculiar sensation of somehow being more connected, more in touch, now I’m back with the hard copy. Of course, at some point the sheer weight of back issues in my office may force me to reconsider, but for now I remain happily wedded to paper.

Memory in peer review: déjà vu all over again

In the periodic debates about the efficacy of the current peer review system for ensuring the quality of the published scientific record, two related topics that come up time and again are the lack of repeatability (or consistency), and the lack of memory. Simply put, the opinions of two or three people constitute a very small sample of the sum of expertise on the subject, so the selection of reviewers can become critical in deciding whether or not a paper is accepted. And a piece of work that gets panned (even for serious technical flaws) in review for one journal, can be submitted, unchanged, to another – and (given a new pair of reviewers) may sneak in with the flaw unspotted on this second occasion. None of this is new, and I stand by my (not exactly controversial) view that the system we have works remarkably well given its unavoidable shortcomings. But sometimes a concrete example comes along to remind one that these arguments are not simply conceptual.

Just over a year ago, I reviewed a paper for a reasonably prestigious biological research journal (IF: 5.064). I thought it was an interesting idea, but rather poorly executed, and I provided what I considered to be a pretty thorough and constructive set of comments which if addressed, would I believe have resulted in a much stronger paper.

Fast forward to this week, and browsing the new content of a different reasonably prestigious journal of integrative biology (IF: 4.736) I see a familiar looking title. There it is: the same piece I reviewed, somewhat modified but with large sections verbatim. Including, for instance, a reference to some of my work in a sentence beginning ‘Likewise…’, where in my review I had written: “‘Likewise’ is almost exactly the wrong word here!”

What to make of it? On one level, well, good luck to them. Of course I think I was right, but they clearly didn’t and gambled on getting a different reviewer in submitting essentially the same work that I’d already criticised. Somewhat frustrating though, in that I think they could have written a better paper had they taken on board my suggestions.

I know some journal families (including Nature I believe) allow reviews to be carried over from one journal to another (lower-ranking!) family member. But if you get a harsh review and decide to submit elsewhere entirely, clearly you are not going to forward the previous review on to the editor. So I guess this quirk will remain with us.

And on this occasion, well, that’s fine. I did, in the end respond the paper that annoyed me last year, and the response is now in press. But I’ve since read that
rebuttals don’t work, so perhaps I should concentrate more on my own work, and spend less time worrying about the errors of others!