Support #2269
open
Multilevel model guidance
50%
Description
Dear Support Team,
Thank you for your comprehensive guidance on the website and in this forum.
I’m currently trying to better understand your guidance on multi-level modelling (https://www.understandingsociety.ac.uk/documentation/mainstage/user-guides/main-survey-user-guide/analysis-guidance-for-weights-when-fitting-multilevel-models/). I’ve also reviewed previous responses https://iserredex.essex.ac.uk/support/issues/1890 and https://iserredex.essex.ac.uk/support/issues/1572, but I’m still finding some aspects unclear:
- In the guidance (example with PSU = level 1, household = level 2, and individual = level 3), the design weight psnenus_xd appears to be applied at the household level, although it is labelled as level 1. In other responses, it seems to be associated with the PSU level. Could you clarify this?
- The derived level 2 weight seems to correspond to the unconditional individual-level weight, yet it is labelled as level 2 rather than level 3. Am I misunderstanding what it done here?
- How would this guidance apply in a longitudinal context (PSU, individual, wave), particularly given that UKHLS already provides longitudinal weights?
- I realise this may not be for this forum, but does the pwscale() option not apply in some models? When trying this with a longitudinal model in Stata, I got an error saying this was not allowed.
Many thanks and best wishes,
Verena
Updated by Verena Schneider 5 days ago
I have forgotten to add one more question:
Dear Support Team,
Thank you for your comprehensive guidance on the website and in this forum.
I’m currently trying to better understand your guidance on multi-level modelling (https://www.understandingsociety.ac.uk/documentation/mainstage/user-guides/main-survey-user-guide/analysis-guidance-for-weights-when-fitting-multilevel-models/). I’ve also reviewed previous responses https://iserredex.essex.ac.uk/support/issues/1890 and https://iserredex.essex.ac.uk/support/issues/1572, but I’m still finding some aspects unclear:
- In the guidance (example with PSU = level 1, household = level 2, and individual = level 3), the design weight psnenus_xd appears to be applied at the household level, although it is labelled as level 1. In other responses, it seems to be associated with the PSU level. Could you clarify this?
- The derived level 2 weight seems to correspond to the unconditional individual-level weight, yet it is labelled as level 2 rather than level 3. Am I misunderstanding what it done here?
- Can I follow this guidance when using both BHPS and UKHLS data? It seems that the weights are composed differently.
- How would this guidance apply in a longitudinal context (PSU, individual, wave), particularly given that UKHLS already provides longitudinal weights?
- I realise this may not be for this forum, but does the pwscale() option not apply in some models? When trying this with a longitudinal model in Stata, I got an error saying this was not allowed.
Many thanks and best wishes,
Verena
Updated by Understanding Society User Support Team 3 days ago
- Status changed from New to In Progress
- Assignee changed from Olena Kaminska to Understanding Society User Support Team
- % Done changed from 0 to 10
- Private changed from Yes to No
Hello Verena,
Many thanks for your enquiry. The Understanding Society team is looking into it and we will get back to you as soon as we can. We aim to respond to simple queries within 48 hours and more complex issues within 7 working days.
Best wishes,
Understanding Society User Support Team
Updated by Understanding Society User Support Team 2 days ago
- Category changed from Survey design to Data documentation
- Status changed from In Progress to Feedback
- % Done changed from 10 to 50
Hello Verena,
Please find below the response from our Weighting Team:
The confusion here (which we will need to clarify in the text) is that the multilevel model itself only has two levels: level 1 (for individuals) and level 2 (for households). This is because the weighting for multilevel modelling literature that we’re aware of is only set up for two-level models. But because UKHLS involves PSU-level clustering at level 3, we propose to account for the impact of level 3 clustering not by including level 3 in the model but by using cluster-robust standard errors. This is a ‘good enough’ solution in lieu of fully developed methodology for 3-level models. We will also clarify the ordering of levels (some packages require level 1 at the lowert, some require level 3 as the lowest) for this example.
So we’re sorry for the confusion but hope this helps. We are grateful for bringing this to our attention and will amend the guidance accordingly
Best wishes,
Understanding Society User Support Team
Updated by Verena Schneider 2 days ago
Dear Support Team,
Many thanks for the clarification. Could I also follow up on the other points?
- In the guidance, the design weight psnenus_xd appears to be applied at the household level. In other responses (e.g., here: https://iserredex.essex.ac.uk/support/issues/1890), it seems to be associated with the PSU level. Could you clarify this?
- Can I follow this guidance when using both BHPS and UKHLS data? It seems that the weights are composed differently for BHPS.
- How would this guidance apply in a longitudinal context (PSU, individual, wave), particularly given that UKHLS already provides longitudinal weights? Would it suffice to use the longitudinal weight at level 2 and include cluster-robust standard errors for PSU?
- I realise this may not be for this forum, but does the pwscale() option not apply in some models? When trying this with a longitudinal model in Stata, I got an error saying this was not allowed.
Many thanks and best wishes,
Verena
Updated by Understanding Society User Support Team about 17 hours ago
Hello Verena,
Please find below the response to questions 3 and 4. I will send the remaining information as soon as I have it.
Bullet point 3:
Although the multilevel models are similar, the situation for longitudinal analysis is different because the weights and levels do not align. The individual (longitudinal) weight is the level 2 weight whereas the lowest level (in this case wave) needs no weight.
As already was mentioned, this guidance only works for two-level because there hasn’t been any methodological work on the more general case (that we’re aware of) since the papers we reference.
Hence, for the highest level, it comes down to the choice between whether to account for the household or the PSU when it comes to standard error estimation; I’d expect that the household would have the bigger impact because the intra-class correlation is going to be larger than for PSUs, so would advise that – but neither will be a feature of the model, just an adjustment for non-i.i.d./simple random sampling.
Bullet point 4:
The pwscale option only works for the simple two-level situations we have described because it is peculiar to the multilevel weighting methods implemented in Mixed. Mixed incorporates scaling of the weights for these cases (the manual explains how https://www.stata.com/manuals13/me.pdf) but the Mixed function allows more complex models than that - so pwscale() won’t work for those (and, we surmise, you fitted a more complex model when you got the warning message).
I hope this information is helpful.
Best wishes,
Understanding Society User Support Team