Understanding Society User Support

Samir,

Thank you for your question. I think you are using pooled analysis putting together waves 1 to 8.
For this using just wave 1 weight would not be the best solution. Instead you would need to extract a cross-sectional weight from each of the waves and match it to the relevant wave. So, for example observations from wave 1 would have wave 1 weight, and observations from wave 2 would have wave 2 weight and so on.

It would be good to scale the weights as per discussion with Evan. But this is less important if you are using only UKHLS. One way to check is to note the total number of respondents in your models in each wave. If this differs substantially you would need this extra scaling. This is important only if you want to represent years evenly. If you are happy for the earlier years to contribute to your analysis more than later years you can skip this scaling factor.

Your points 5) and 6) are correct but are not relevant. You will have most observations if you use the method and weights above.

Your point 7) is correct. Yet, your sample size would go up if you start with wave 2 and you use weight ub instead of us; or from wave 6 you can use ui weight.

Thank you,
Olena

Hello Olena,

Thanks a lot, this is v helpful.

As discussed, I applied the changes but I wanted to check I used the correct weights. Can you confirm the below is correct?

apply the 'a_indinus_xw' weight for all observations in wave 1
apply the 'b_indinub_xw' weight for all observations in wave 2
apply the 'c_indinub_xw' weight for all observations in wave 3
apply the 'd_indinub_xw' weight for all observations in wave 4
apply the 'e_indinub_xw' weight for all observations in wave 5
apply the 'f_indinui_xw' weight for all observations in wave 6
apply the 'g_indinui_xw' weight for all observations in wave 7
apply the 'h_indinui_xw' weight for all observations in wave 8

If the above is correct, I have a follow on question. Once I set svyset (see below code), I get an error message in Stata (see below) when I run my multilevel model seemingly because the weights within pidps are not consistent. I have run the model with the cross-section weights (as described above) without taking 'strata' and 'psu' into account so there is no issue there, but I would be keen to take the complex survey design into account to get more precise standard errors. Are you by any chance aware of how that could be done?

That said, even if I did account for 'strata' and 'psu' I wouldn't expect the change in SE (which would be an increase, correct?) to be that large, and it would have no effect on the covariate coefficients (am I right?), meaning that the weighted model alone (i.e., without taking the complex survey design into account) is satisfactory. Is my reasoning correct?

Code:
svyset, clear
svyset psu, strata(strata) weight(final_weight)

Stata Error Message:
'weights in variable final_weight not constant within groups defined by: pidp an error occurred when svy executed melogit'

As always, thanks for your ongoing support Olena.

I look forward to hearing from you.

With very best wishes,

Samir

Regarding my last question, I am essentially trying to understand the consequences on my model/results of not taking the complex survey design (i.e., strata and psu) into account.

Thank you.

Samir,

Thank you for your question. I am not very familiar with melogit command in Stata, but looking at the Stata help, I would suggest you try the following svyset commands:

1) make sure your data is in long format.

2) try the Stata suggested svyset command:

svyset psu [pw=final_weight], strata(strata)
By the way you should be able to get psu and strata variables from xwavedat that's ready for you, i.e. you wouldn't need to extract them from all the waves.

3) If the above doesn't work, try the following:

svyset psu || pidp, weight(final_weight)
svy: melogit y x || psu: || ssu:

My understanding is that psu should be as a psu in our dataset, and ssu is pidp.

This way you will not take into account stratification making your estimates conservative.

With regard to your last question not taking into account stratification is fine - your results are just more conservative. But you should take into account psu and weights - both, point estimates and relationships are likely to be influenced.

Hope this helps,
Olena

Hello Olena,

Thanks a lot for this.

Unfortunately both options suggested didn't work. More specifically, using "svyset psu [pw=final_weight], strata(strata)" generates an error message that reads:

"survey final weights not allowed with multilevel models; a final weight variable was svyset using the [pw=exp] syntax, but multilevel models require that each stage-level weight variable is svyset using the stage's corresponding weight() option an error occurred when svy executed melogit."

Meanwhile, running:

"svyset psu || pidp, weight(final_weight)
svy: melogit y x || psu: || ssu:"

generates an error message that reads:

"weights in variable final_weight not constant within groups defined by: psu pidp an error occurred when svy executed melogit."

I half expected that to be honest because the 'final_weight' has got cross-sectional waves so that the same pidp has different weights depending on the year/wave it relates to (i.e., 1 or 2, or 3 etc...)

That said, my thinking was that it should make sense to run the model as:

"svyset psu || pidp, weight(final_weight)
svy: melogit y x || pidp"

That is, signaling 'psu' through the svyset specification and letting the melogit command take care of the clustering of observations at the individual level through the '|| pidp' specification.

Would you be able to confirm if this understanding is correct or if it would be more appropriate to run the model as

"svyset psu || pidp, weight(final_weight)
svy: melogit y x || psu: "

As always, I appreciate your continued help with this.

With very best wishes,

Samir

Hi Samir,

This issue is to do with how melogit works in Stata. My suggestion would be to look at Statalist and see if others have asked and answered this issue. If not, you can post an issue on Statalist and see what they have to say.

For example, here is an issue posted on Statalist about melogit and weights:
https://www.statalist.org/forums/forum/general-stata-discussion/general/1344286-problem-applying-sampling-weights-for-two-level-mixed-effects-logit-melogit

Having said that I had a look at melogit help in Stata. For pweights it said " pweight(varname) sampling weights at higher levels". This means that if your two levels are person (pidp) and time (wave), then pidp is the higher level and so Stata expects there to be a weight at the individual level only. Which means it should be the longitudinal weight from the last wave. But please ask about this on Statalist and confirm.

If you have questions about which weight to use, or need data managgement guidance in order to be able to use the weights, etc please post your question and we will get back to you. If you want to join the Helpdesk Hour tomorrow and ask Olena further questions on weights, please email usersupport@understandingsociety.ac.uk

Best wishes,
Alita

Dear Alita,

Thanks a lot for this.

This is v helpful and in line with my understanding from the readings. I had previously reached out via Statalist but hadn't had much luck - I'll try again and hopefully have more success this time.

Just one small point to clarify when you say 'which means it should be the longitudinal weight from the last wave', you mean in the case of if I was looking at undertaking a balanced panel data analysis? But in context of what I previously discussed above with Olena, such a weight could also be the cross-sectional one previously suggested for an unbalanced analysis. Correct?

Thanks a lot,

Samir

The question you need to ask is why are you using weights. If your analysis is longitudinal (FE, RE, multi-level modelling where the 2 levels are person and time) then you should use longitudinal weights as it will account for non-response at wave 1 + attrition over the years. The only reason you have to use balanced panel is that we don’t provide weights for any other combination of waves. So, you can either use the longitudinal weights we provide with balanced panel, or produce longitudinal weights which you can use with unbalanced panel data.

If on the other hand you are doing pooled cross-sectional analysis (e.g., pooled OLS) then you should use cross-sectional weights from all the waves as Olena suggested.

Hope this helps,
Alita

Thanks a lot Alita - v helpful. Please also forward my thanks to Olena for her help throughout.

Have a pleasant day.

Samir