Chapter 4: Weeks 5 and 6

I had scratched my head in every way possible. However, I was no closer to actually debugging the error, which perplexed me. I searched every relevant function, every relevant block of code, understood how each and every parameter was being calculated, and made a mental map of how functions were being accessed and in which order. I was still stuck. And so, I decided to just scrap my current implementation, and started over. I had the chunksize method implemented for non-optimized loops for reference. The error, it turned out, was much simpler than what I’d made of it. Initially, I was updating the parameters through two different loops, each iterating over the dataframe in chunks. After that, the second loop was being run, that basically applied the broadening. Both these processes went hand-in-hand, however I was separating them, which implied incorrect results.

Hence, I fixed the error and began the testing process. It was around this time that I also mentioned to my mentors about restructuring my timeline a bit, such that it would incorporate the proposed deliverables in a more realistic timeline. So far, I hadn’t been able to deliver on a lot of the work that I promised I’d do, because I was actually managing two internships at the same time. Thanks to my very understanding mentors, I could get the timeline structured better!

I then wrote a test function in test_broadening.py, which compared different chunksize values and made sure that the residual between the equilibrium spectra calculated by using these values was negligible at best. I was able to get a satisfactory result after some modifications and suggestions by Dr. Erwan. I also added an example in the help section in my function.

Apart from that, I also created a benchmarking notebook for comparing the different outputs ann their calculation times, produced by the calculation of equilibrium spectra based on different chunksize methods. I was in close touch with Mr. Minou and Dr. Erwan during this period, where I created a plot that was crucial to the benchmarking process - a plot of Chunksize vs. Calculation Times.

This plot was the embodiment of my work upto now; the work that had reduced the memory usage of calculating an equilibrium spectrum using RADIS, while also trying to have it fast enough. And sure enough, the graph showed great results! There was proof that introducing the chunksize method was actually efficient while consuming lesser memory. My mentors ensured that I had great plotting practice!