Kaja Milczewska – k.m.milczewska@pgr.reading.ac.uk

Completing a PhD is a momentous task at the best of times, let alone in combination with a year-long global pandemic. Every PhD researcher is different, and as such, everyone has had different circumstantial struggles throughout Covid-19. The lack of human interaction that comes with working in a vibrant academic environment such as the Meteorology Department can make working from home a real struggle. Sometimes it is difficult to find the motivation to get anything useful done; whereas at other times you could squeeze five hours’ worth of work into one. Trying to stay organised is key to getting it done, therefore the following are some of the things that helped me get to the end of my PhD thesis – and it has not been easy. If you are still out there writing and finishing up experiments: read on! Maybe the result is that you might feel a little less alone. The PhD experience can be truly isolating at the best of times, so literally being instructed to isolate from the world is not ideal. The points are numbered for convenience of structuring this post, rather than any order of importance. 

1. Communicate with your supervisor(s) 

It is tempting to “disappear off the radar” when things are not going well. You could wake up in the morning of the day of your regular weekly meeting, filled with dread that you have not prepared anything for it. Your brain recoils into the depths of your skull as your body recoils back under the safety of the duvet. What are your options? Some of them might be: take a deep gulp and force yourself out of bed with the prospect of coffee before the meeting (where you steer the conversation onto the things you did manage to do); or to postpone the meeting because you need to finish XYZ and thus a later meeting may be more productive; or ignore the meeting altogether. The first one is probably the best option, but it requires mental strength where there might be none. The second one is OK, but you still need to do the work. The last one is a big no. Don’t do it. 

Anxiety will make you believe that ignoring the world and all responsibilities is the most comfortable option in the moment, but the consequences of acting on it could be worse. Supervisors value honesty, and they know well that it is not always possible to complete all the scheduled tasks. Of course, if this happens every week then you might need to introspectively address the reasons for this, and – again, talking with your supervisor is usually a useful thing to do. You might not want them to know your entire life story, but it is helpful for everybody involved if they are aware that you struggle with anxiety / depression / ADHD / *insert any condition here*, which could affect your capacity to complete even the simplest, daily tasks. Being on the same page and having matching expectations is key to any student – supervisor partnership. 

2.  Reward yourself for the things you have already accomplished. 

Whether that’s mid-week, mid-to-do-list, weekend — whenever. List all the things you have done regularly (either work- or life-related) and recognise that you are trying to survive a pandemic. And trying to complete the monstrous task of writing a PhD thesis. Those are big asks, and the only way to get through them is to break them down into smaller chunks. Putting down “Write thesis” on your to-do list is more likely to intimidate than motivate you. How about breaking it down further: “Re-create plot 4.21”, or “Consolidate supervisor comments on pages 21 – 25” — these are achievable things in a specified length of time. It also means you could tick them off more easily, hopefully resulting in feeling accomplished. Each time this happens, reward yourself in whatever way makes you feel nice. Even just giving yourself a literal pat on the shoulder could feel great – try it! 

3. Compile supervisor feedback / comments into a spreadsheet  

An Excel spreadsheet – or any other suitable system – will enable you to keep track of what still needs addressing and what has been completed. The beauty of using a colour-coded spreadsheet for feedback comments is that once the required corrections are completed, you have concrete evidence of how much you have already achieved – something to consult if you start feeling inadequate at any point (see previous section!). I found this a much easier system than writing it down in my workbook, although of course this does work for some people, too. Anytime you receive feedback on your work – written or otherwise – note them down. I used brief reminders, such as “See supervisor’s comment on page X” but it was useful to have them all compiled together. Also, I found it useful to classify the comments into ‘writing-type’ corrections and ‘more work required’ corrections. The first one is self-explanatory: these were typos, wrong terminologies, mistakes in equations and minor structural changes. The ‘more work required’ was anything that required me to find citations / literature, major structural changes, issues with my scientific arguments or anything else that required more thought. This meant that if my motivation was lacking, I could turn to the “writing-type” comments and work on them without needing too much brain power. It also meant that I could prioritise the major comments first, which made working to a deadline a little bit easier. 

4. Break down how long specific things will take 

This is most useful when you are a few weeks away from submission date. With only 5 weeks left, my colour-coded charts were full of outstanding comments; neither my ‘Conclusions’ chapter nor my Abstract had been written; plots needed re-plotting and I still did not know the title of my thesis. Naturally, I was panicking. I knew that the only way I could get through this was to set a schedule — and stick to it. At the time, there were 5 major things to do: complete a final version of each of my 5 thesis chapters. A natural split was to allow each chapter only one week for completion. If I was near to running over my self-prescribed deadline, I would prioritise only the major corrections. If still not done by the end of the allowed week: that’s it! Move on. This can be difficult for any perfectionists out there, but by this point the PhD has definitely taught me that “done” is better than perfect. I also found that some chapters took less time to finish than others, so I had time to return to the things I left not quite finished. Trust yourself, and give it your best. By all means, push through the hardest bit to the end, but remember that there (probably) does not exist a single PhD thesis without any mistakes. 

5. Follow useful Twitter threads 

There exist two groups of people: those who turn off or deactivate all social media when they need to focus on a deadline, and those who get even more absorbed by its ability to divert your attention away from the discomfort of the dreaded task at hand. Some might call it “productive procrastination”. I actually found that social media helped me a little – but only when my state of mind was such that I could resist the urge to fall down a scrolling rabbit hole. If you are on Twitter, you might find hashtags like #phdchat and accounts such as @AcademicChatter , @phdforum @phdvoice useful. 

6. Join a virtual “writing room” 

On the back of the last tip, I have found a virtual writing room helpful for focus. The idea is that you join an organised Zoom meeting full of other PhDs, all of whom are writing at the same time. All microphones are muted, but the chat is active so it is nice to say ‘hello!’ to someone else writing at the same time, anywhere else in the world. The meetings have scheduled breaks, with the organiser announcing when they occur. I found that because I actively chose to be up and start writing at the very early hour of 6am by attending the virtual writing room, I was not going to allow myself to procrastinate. The commitment to being up so early and being in a room full of people also doing the same thing (but virtually, obviously) meant that those were the times that I was probably the most focused. These kinds of rooms are often hosted by @PhDForum on Twitter; there could also be others. An alternative idea could be to set up a “writing meeting” with your group of peers and agree to keep chatter to a minimum (although this is not something I tried myself). 

7. Don’t look at the news 

Or at least, minimise your exposure to them. It is generally a good thing to stay on top of current events, but the final stages of writing a PhD thesis are probably unlike any other time in your life. You need the space and energy to think deeply about your own work right now. Unfortunately, I learnt this the hard way and found that there were days where I could do very little work because my brain was preoccupied with awful events happening around the world. It made me feel pathetic, routinely resulting in staying up late to try and finish whatever I failed to finish during the day. This only deteriorated my wellbeing further with shortened sleep and a constant sense of playing “catch-up”. If this sounds like you, then try switching off your news notifications on your phone or computer, or limit yourself to only checking the news homepage once a day at a designated time.  

8. Be honest when asked about how you are feeling 

Many of us tend to downplay or dismiss our emotions. It can be appealing to keep your feelings to yourself, saving yourself the energy involved in explaining the situation to whomever asked. You might also think that you are saving someone else the hassle of worrying about you. The trouble is that if we continuously paper over the cracks in our mental wellbeing within the handful of conversations we are having (which are especially limited during the pandemic), we could stop acknowledging how we truly feel. This does not necessarily mean spilling all the beans to whomever asked the innocent question, “How are you?”. But the catharsis from opening up to someone and acknowledging that things are not quite right could really offload some weight off your shoulders. If the person on the other end is your PhD supervisor, it can also be helpful for them to know that you are having a terrible time and are therefore unable to complete tasks to your best ability. Submission anxiety can be crippling for some people in the final few weeks, and your supervisor just won’t be able to (and shouldn’t) blindly assume how your mental health is being affected by it, because everyone experiences things differently. This goes back to bullet no.1. 

Hopefully it goes without saying that the above are simply some things that helped me through to the end of the thesis, but everybody is different. I am no counsellor or wellbeing guru; just a recently-finished PhD! Hopefully the above points might offer a little bit of light for anyone else struggling through the storm of that final write-up. Keep your chin up and, as Dory says: just keep swimming. Good luck! 

James Fallon – j.fallon@pgr.reading.ac.uk

As researchers, we familiarise ourselves with many different datasets. Depending on who put together the dataset, the variable names and definitions that we are already familiar from one dataset may be different in another. These differences can range from subtle annoyances to large structural differences, and it’s not always immediately obvious how best to handle them.

One dataset might be on an hourly time-index, and the other daily. The grid points which tell us the geographic location of data points may be spaced at different intervals, or use entirely different co-ordinate systems!

However most modern datasets come with hidden help in the form of metadata – this information should tell us how the data is to be used, and with the right choice of python modules we can use the metadata to automatically work with different datasets whilst avoiding conversion headaches.

First attempt…

Starting my PhD, my favourite (naïve, inefficient, bug prone,… ) method of reading data with python was with use of the built-in function open() or numpy functions like genfromtxt(). These are quick to set up, and can be good enough. But as soon as we are using data with more than one field, complex coordinates and calendar indexes, or more than one dataset, this line of programming becomes unwieldy and disorderly!

>>> header = np.genfromtxt(fname, delimiter=',', dtype='str', max_rows=1)
>>> print(header)
['Year' 'Month' 'Day' 'Electricity_Demand']
>>> data = np.genfromtxt(fnam, delimiter=',', skip_header=1)
>>> print(data)
array([[2.010e+03, 1.000e+00, 1.000e+00, 0.000e+00],
       [2.010e+03, 1.000e+00, 2.000e+00, 0.000e+00],
       [2.010e+03, 1.000e+00, 3.000e+00, 0.000e+00],
       ...,
       [2.015e+03, 1.200e+01, 2.900e+01, 5.850e+00],
       [2.015e+03, 1.200e+01, 3.000e+01, 6.090e+00],
       [2.015e+03, 1.200e+01, 3.100e+01, 6.040e+00]])

The above code reads in year, month, day data in the first 3 columns, and Electricity_Demand in the last column.

You might be familiar with such a workflow – perhaps you have refined it down to a fine art!

In many cases this is sufficient for what we need, but making use of already available metadata can make the data more readable, and easier to operate on when it comes to complicated collocation and statistics.

Enter pandas!

Pandas

In the previous example, we read in our data to numpy arrays. Numpy arrays are very useful, because they store data more efficiently than a regular python list, they are easier to index, and have many built in operations from simple addition to niche linear algebra techniques.

We stored column labels in an array called header, but this means our metadata has to be handled separately from our data. The dates are stored in three different columns alongside the data – but what if we want to perform an operation on just the data (for example add 5 to every value). It is technically possible but awkward and dangerous – if the column index changes in future our code might break! We are probably better splitting the dates into another separate array, but that means more work to record the column headers, and an increasing number of python variables to keep track of.

Using pandas, we can store all of this information in a single object, and using relevant datatypes:

>>> data = pd.read_csv(fname, parse_dates=[['Year', 'Month', 'Day']], index_col=0)
>>> data
Electricity_Demand
Year_Month_Day      
2010-01-01      0.00
2010-01-02      0.00
2010-01-03      0.00
2010-01-04      0.00
2010-01-05      0.00
...              ...
2015-12-27      5.70
2015-12-28      5.65
2015-12-29      5.85
2015-12-30      6.09
2015-12-31      6.04

[2191 rows x 1 columns]

This may not immediately appear a whole lot different to what we had earlier, but notice the dates are now saved in datetime format, whilst being tied to the data Electricity_Demand. If we want to index the data, we can simultaneously index the time-index without any further code (and possible mistakes leading to errors).

Pandas also makes it really simple to perform some complicated operations. In this example, I am only dealing with one field (Electricity_Demand), but this works with 10, 100, 1000 or more columns!

• Flip columns with data.T
• Calculate quantiles with data.quantile
• Cut to between dates, eg. data.loc['2010-02-03':'2011-01-05']
• Calculate 7-day rolling mean: data.rolling(7).mean()

We can insert new columns, remove old ones, change the index, perform complex slices, and all the metadata stays stuck to our data!

Whilst pandas does have many maths functions built in, if need-be we can also export directly to numpy using numpy.array(data['Electricity_Demand']) or data.to_numpy().

Pandas can also simplify plotting – particularly convenient when you just want to quickly visualise data without writing import matplotlib.pyplot as plt and other boilerplate code. In this example, I plot my data alongside its 7-day rolling mean:

ax = data.loc['2010'].plot(label='Demand', ylabel='Demand (GW)')
data.loc['2010'].rolling(7).mean().plot(ax=ax, label='Demand rolling mean')
ax.legend()

Now I can visualise the anomalous values at the start of the dataset, a consistent annual trend, a diurnal cycle, and fairly consistent behaviour week to week.

Big datasets

Pandas can read from and write to many different data formats – CSV, HTML, EXCEL, … but some filetypes like netCDF4 that meteorologists like working with aren’t built in.

xarray is an extremely versatile tool that can read in many formats including netCDF, GRIB. As well as having built in functions to export to pandas, xarray is completely capable of handling metadata on its own, and many researchers work directly with objects such as xarray DataArray objects.

There are more xarray features than stars in the universe^{[citation needed]}, but some that I find invaluable include:

open_mfdataset – automatically merge multiple files (eg. for different dates or locations)
assign_coords – replace one co-ordinate system with another
where – replace xarray values depending on a condition

Yes you can do all of this with pandas or numpy. But you can pass metadata attributes as arguments, for example we can get the latitude average with my_data.mean('latitude'). No need to work in indexes and hardcoded values – xarray can do all the heavy lifting for you!

Have more useful tips for working effectively with meteorological data? Leave a comment here or send me an email j.fallon@pgr.reading.ac.uk 🙂