Rubber Ducky

[da]

Today was a whole pile of work, and I loved it.

First off I had a chat with my boss, including a bit about a project I'll be advising them for starting on Monday. She said, "Why don't I show you the mice?" We went down the hall and into a lab housing a dozen white mice. (They're genetic knockouts for a particular bone-regrowth gene, and they're studying whether they have a higher, or lower, metabolism because they aren't making that particular protein. I think- it was explained rather rapidly.) Their boxes were sealed to the outside and had sensors for oxygen-use, food consumption, and a batch of security-system tripwire lasers in each box to measure their activity-levels. The data got dumped into a machine which... they were reading in an excel spreadsheet. Monday, they're going to ask me how to divide the spreadsheet data into time intervals. It all looked quite professional, and I hope it's as simple a problem as they actually described. (Of course it won't be; there will be exceptions. There always are.)

Shortly afterward, I got an email from a grad student in a spine-lab who had an easy(ish) Visual Basic question, and a difficult LabView question. We made plans to meet after lunch.

Lunch was the Department Christmas Party at the Faculty Club. I sat next to a charming and vaguely eccentric retired professor who lives out in the boonies and is getting Ontario Power to put a generating windmill on his property... The two administrative staff sitting on the other side of the table were getting a bit wide-eyed at some of the things he was over-sharing about (which I won't indulge in here, thanks much). I enjoyed the lunch and conversation, then excused myself at 1:30 to meet with the student.

The meeting was dizzying. This project involves a... robotic bench-press, sort of, which compresses and moves sections of spine (!) and measures the resultant forces in various directions. Her labview program is complex, but she thinks it should be able to sample at least four times faster. So I sat there for a while going "uh-huh" as she told me piles more information than I could understand at my current level... I was tremendously amused that just last night, bats22 told melted_snowball and me about the Rubber Ducking technique, and here I was, a professional rubber duck. I did contribute a few general suggestions ("Can we break the problem into smaller problems? Can we move other stuff outside the loops?") and by the time I walked out, she had a pile of new things to try. This is precisely what I think I was hired for, and I hope to be able to gradually increase the usefulness of my comments. But boy do I have a lot to learn.

Hm, I was going to write all about my new to-do system, but I think that will wait.

Hey. I seem to have hit the ground running. I'm looking forward to tomorrow!

Comments

[dr-tectonic.livejournal.com]

So what's your first thoughts about how to approach the Excel problem? I'm curious to see if they're at all like what my instinct is.

[da-lj.livejournal.com]

I didn't fully describe the inputs: there's a column with a custom-formatted date and time, so I can just reformat that as a proper date/time and do date-compares to get the right rows. That part should be easy; even if there wasn't a column with time-data, it's all going to be at predetermined intervals so we could just step n rows.

Mostly, my thoughts are figuring out what I need to ask: what are they doing with the results; to help determine how would they like their time-intervals separated (dumped to separate excel files, separate worksheets, highlighted piece-by-piece when they click a button, or something else?)

For now I'm assuming they don't need further data-massaging, but I bet that'll be coming along as a later step.

I'm curious what sorts of things this made you think about!

[dr-tectonic.livejournal.com]

Oh, mostly just that if I needed to do some transformations on Excel data that were at all complicated, my first instinct would be to dump it to a text file, massage it in perl, and then reload it into Excel. I was imagining things like the timestamps being spread across multiple columns in an unhelpful way, or that you had to combine a time cell with the filename, or that they wanted you to separate the data on different days into different files/sheets or something like that. Hooray for just needing to reformat a column!

[da-lj.livejournal.com]

Yah, in that situation, I'd get out the swiss-army chainsaw that is Perl. :)

Spreadsheet::ReadExcel has at times made my life much smoother, when clients have insisted on handing off an excel spreadsheet instead of a .csv file...

[ng-nighthawk.livejournal.com]

Of course it won't be; there will be exceptions. There always are.

A lot of this, in my experience, is the natural inclination to realize that you can do more. When you're barely stumbling along, you have very simple goals because that's all you can have. Once you have accomplished that simple goal, you realize there are many other things which could be done.

[da-lj.livejournal.com]

To my mind, that's a second, often desirable thing: iterative design can even be a goal (as long as the client understands that it might be a re-design not "just a tweak").

But for now, I'm just whinging that they didn't get to tell me the full spread of what they want initially, just a brief intro, and I can't possibly start writing (or really thinking about design) until the first real meeting on Monday.

[sulle-stelle.livejournal.com]

I ditto the recommendation of massaging in perl if the manipulation is too complex, because doing it in Excel can be dog-slow. I ran into this issue in my data mining class. If you must stay inside Excel, I'd recommend the "step n" approach as opposed to the date conversion.