Understanding Society User Support

Marina,

You could use calendar data longitudinally with a correct set up. For this you would need to select the weight from the last wave in your analysis for everyone. So, if the last wave, even for a few people, is wave 13, you should select longitudinal weight from wave 13 for everyone. This way you will be representing the (sub)population for 13 waves, and values from different waves (but same years) will be seen as their characteristics.

You could use interview date, if it is important, but the calendar year needs to have everyone responding in it (technically from 3 waves), as described in the relevant question here (that I can't access at the moment):
https://www.understandingsociety.ac.uk/sites/default/files/downloads/general/weighting_faqs.pdf

Hope this helps,
Olena

Hi Olena,

Many thanks for your very useful reply. May I ask two follow-up questions:

1) I was under the impression that to create a calendar year correctly and include everyone who responded, I only need to use two waves, e.g. to create a 2019 calendar year dataset I would need to combine W10 (year 2) and W11 (year 1) using the w_month variable. Apologies, but the link provided above does not seem to be working.

2) I saw elsewhere on this forum that I need to have balanced panels for the longitudinal weights to work properly; What are the effects on my analysis if I am using the Understanding society longitudinal weights with unbalanced data? E.g. I have waves 7-10 but not all pidp appear in all waves. OR in the case where e.g. I am using waves 7 , 9, 11 (so I am skipping waves 8 and 10).

3) In the case that I am really not allowed to use the provided weights on unbalanced data (e.g. for waves 9-13) how could I edit the weights (including psu and strata to adjust for the complex survey design) to accommodate an appropriate analysis with unbalanced data?

Many thanks again for your help.

Best wishes,
Marina

Marina,

Yes, it is usually 3 waves to create one calendar year. Read more here: question 13 https://www.understandingsociety.ac.uk/wp-content/uploads/working-papers/2024-01.pdf .

Our longitudinal weights can be used with unbalanced analysis - but they are created for balanced, so the sample size would be smaller than for unbalanced. You could create your own tailored weight as here: https://www.understandingsociety.ac.uk/help/training/creating-tailored-weights/

Variables such as strata and psu will not change with your type of analysis and model - they are stable. Always use them in your analysis.

Hope this helps,
Olena

Massive thanks for all of your help! The materials you sent me appear to be very useful and will read them in detail.

I have a few final (hopefully) questions. I can see also the first pdf you recommended seems to partly shed some light on a few issues I am facing. However, I wanted to further ask for your support if possible. I will also describe in more detail what I am trying to do as it may help too.

1) I remain a bit confused about deriving a correct calendar year from three waves using the suggested w_month, since in all waves I looked at there are only months 1-24 (assuming months 1-12 correspond to sample year 1, and 13-24 to sample year 2 in each wave). So I do not understand how it is possible to extract the calendar year data from three waves using exclusively the sample year/month variables, since these only seem to reflect two years. This was another reason why I was thinking I could use the interview date instead. But please let me know if I am misunderstanding something here,

2) In more detail, I wish to perform a diff-in-diff (DiD)analysis for a policy evaluation. The policy was implemented in March 2020 but was subsequently reverted in September 2021. So I wish to have a pre-treat period that comes before March 2020 and the post-treat is from March 2020 onwards (but only until September 2021 when the policy is removed again). In this case, I only care to use the interview date to properly define my pre/post variable. But I was a bit confused because I read in some user guides that to ensure all different samples are properly represented in each calendar year,(e.g. NI over-represented etc ) I should aim to compile all relevant calendar year data. But is it actually something I do not need to worry about, considering I will make use of the full waves 9-13, however, I will exclude only those who interviewed after September 2021 in wave 13? I understand this might hurt my analysis (because of the different samples like NI etc of Understanding Society) since I will be excluding a few people from different waves (prob waves 12-13) who had an interview after Sept 2021. But is this something so severe you would advise me to refrain from this analysis altogether? And in this case, would it not still be ok for me to use the longitudinal weight provided in Wave 13?

3) Similarly I am also exploring a second paper where I am instead doing a DiD on the removal of the policy, so my pre-treat period is from March 2020 to September 2021 and my post-treat period is after September 2021, where I will be facing similar issues due to the different samples by sample month/year.

I would greatly appreciate any advice and suggestions you may have on how to proceed.

Best wishes,
Marina

Marina,

I think it will be much clearer once you start setting up your data.
Our data has issue_months 1-24. This is when the sample is issued, but we give them up to 4 months for refusal conversion. Hence the extension over potentially 3+ calendar months for each issue month.
Your rule should be to gather data based on interview date (not issue date), from all waves. Don't miss any, even if you have to go 1-2 waves back. The idea is that very late respondents from previous waves will compensate for late respondents from this issue month that responded later (it is issue months that are representative of a population). And while it may be confusing, your only task is to gather everyone who was interviewed in this particular calendar month / period. The rest is in pdf.

You are also lucky that we expected an interest in before and after March 2020, and model nonresponse separately. So, nonresponse correction is even tailored to your task.

Best of luck,
Olena