Wednesday, July 12, 2006

A day filled with data

Have you ever stared at data so long that it gave you a headache?

Yeah, it was one of those days. Finishing up the data analysis so that we can meet a fast-approaching deadline, we were derailed a bit when we discovered a small error in our analysis. Luckily, it was very minor, and we were able to rerun the necessary experiments quickly. Unfortunately, the new analysis shows that our results are not as good as they were before. They're still good, but it's not a slam-dunk. But there are definitely some new and interesting trends in the corrected data. The problem is that there's something there, some worthwhile insight, in the data that I can intuitively sense but that is not 100% clear to me. Hence, spending most of the day staring at data, replotting it different ways, calculating different errors, printing out graphs and tables, hoping to grasp that crucial insight that I know is there but is so far eluding me. I was hoping to finish this section of the paper today, but I reached a point where I just couldn't think straight anymore, so I went home. Ah, summer.

I'm not too worried at this point--the insight will either come to me, or else I'll come up with some other "good enough" explanation to meet the deadline. Sometimes patience is your best friend when dealing with tricky data. My undergrads, though, are completely mortified. I think I spent as much time today giving them pep talks: no, this is not entirely your fault; no, this does not mean that everything you did this summer is worthless; no, having your hypothesis partially disproved is not the end of the world, and in fact most of the time this leads to the most interesting results. I suspect that I'll have cheerleader duty again tomorrow.

What this experience made me realize is how much of a teacher experience is. I can remember as an undergrad being *terrified* of messing up--that if I made a mistake, I'd be fired from the lab. Even as a grad student, early on, I was afraid of the wrong turn, the dead end, the experiment that didn't work out as planned. Time and experience taught me to value these experiences, to look for the dead ends, to pull something out of even the most badly botched experiments. This is not something any one person taught me, and I know that nothing I say will help my students learn this lesson either. As with my data, the only thing I can do to help my students is to patiently repeat the message that everything will turn out fine in the end, in the hopes that someday much later, they'll understand what I meant.

9 comments:

Ianqui said...

My data doesn't so much give me a headache, but it certainly does give me repetitive stress injury.

Clyde said...

And they will remember your cheerleading the next time they make a mistake in a new context. I, too, remember feeling awful about making mistakes and luckily have had some mentors who talked about just the things you said. Still learning those lessons too. Have more confidence in some areas than others. Congrats on your progress!

Anonymous said...

Wow, thank you for this post. As a grad student working on her first major independent research project, which happens to require lots of contextual and subjective decisions, I am *terrified* that someone is going to find a mistake in my code. I think that's why I'm taking longer to finish than I should -- the longer it takes to complete, the longer I have until it's out there to be scrutinized by the wild internets!

I miss the days when I was just handed a final data set to work with! :)

Dr. Shellie said...

Part of it is adjusting to the long time frame needed to get results in research-- your undergrads probably can't imagine that their work could be contributing to a YEARS-long "project" of gaining knowledge... Keep up the good work cheerleading!

Jane said...

Dr. Shellie, you hit the nail on the head. The biggest struggle my undergrads have had this summer is realizing that research is non-linear. They came in assuming that research is like a big homework assignment--you do tasks X, Y, and Z, and you're done and there's *the* correct answer. It's terrifying to them when they realize that that's not the case at all!

Anon, I don't think you ever completely get over the fear---every time I submit a paper somewhere, I get this overwhelming feeling of dread that *this* is when "they" finally figure out that I have no idea what I'm doing! But I guess the fear does lessen a bit with time. Good luck on your first big project!!

Clyde, thanks! I hope they do remember both the lesson and the pep talk down the road....

Ianqui, that's the beauty of hiring undergrads---either they get the RSI, or they figure out a way to automate everything. :) Either way, it's a win-win for the faculty member!

Wicked Teacher of the West said...

Of course, for new programmers, even responding to homework assignments brings on the same fear - even if it works, what if it wasn't the "right" algorithm? What if I solved it the wrong way?

On the other side, isn't it great when you finally reach the point you describe in your third paragraph? Where you know that either insight will appear or you'll come up with something good enough? And really, I trust that insight will appear - like the zone, it magically appears.

Jane said...

GK, my intro students are definitely guilty of the "but what's the RIGHT way to do it?" mentality. As if their instinct is immediately incorrect...! (and the insight did appear the next day--yay!)

Anonymous said...

We will feel that all our efforts put into this writing about data have not gone to vain if you get some benefit from reading it. Do wish you were benefited.

Anonymous said...

runescape money runescape gold runescape money runescape gold wow power leveling wow powerleveling Warcraft Power Leveling Warcraft PowerLeveling buy runescape gold buy runescape money runescape items runescape gold runescape money runescape accounts runescape gp dofus kamas buy dofus kamas Guild Wars Gold buy Guild Wars Gold lotro gold buy lotro gold lotro gold buy lotro gold lotro gold buy lotro gold runescape money runescape power leveling runescape money runescape gold dofus kamas cheap runescape money cheap runescape gold Hellgate Palladium Hellgate London Palladium Hellgate money Tabula Rasa gold tabula rasa money lotro gold buy lotro gold Tabula Rasa Credit Tabula Rasa Credits Hellgate gold Hellgate London gold dofus kamas buy dofus kamas 血管瘤 肝血管瘤 音乐剧 北京富码电视 富码电视 富码电视台 7天酒店 7天连锁酒店 7天连锁 自清洗过滤器 过滤器 压力开关 压力传感器 流量开关 流量计 液位计 液位开关 温湿度记录仪 风速仪 可燃气体检测仪 wow power leveling wow powerleveling Warcraft PowerLeveling Warcraft Power Leveling World of Warcraft PowerLeveling World of Warcraft Power Leveling runescape power leveling runescape powerleveling