Today, from Melbourne, I watched my friend, colleague and former (50%) PhD student Bin Yu being inducted into the US National Academy of Sciences in Washington DC. As a former President of the IMS, Bin Yu will be familiar to all of you, and I am sure you all shared my pleasure and satisfaction in this recognition of her research achievements. The time we worked together 28 years ago ranks among the happiest of my professional career. I wondered what it was that brought us together, and concluded that it was ideas.

Not long after I joined the Berkeley statistics faculty in 1987, I gave a talk about Jorma Rissanen’s notion of minimum description length (MDL) and its use in model selection. At the time Bin was working with Lucien Le Cam on a topic in empirical processes, and doing very well with it, but she showed an interest in MDL. Together we began reading Rissanen’s Annals of Statistics paper on stochastic complexity, which had appeared the year before. We also met up with Jorma, who lived and worked in nearby San José. Soon Bin was hooked on information theory in general and MDL in particular, beginning her many contributions to these topics.

My awareness of information theory went back to my first year at the University of Melbourne. I used to spend time between classes and labs in the library, mainly people-watching, but also dipping into books. One which fascinated me, to which I returned again and again, was Norbert Wiener’s Cybernetics.

There were so many ideas in that book, I’d put it back on the shelf with my head spinning. Fourier analysis, group theory, ergodic theory, statistical mechanics, Brownian motion, prediction, computers, communication, information, language, psychopathology… The list went on and on, all in one book. This was heady stuff for a freshman science student. I used to wonder whether the time would ever come when I could understand the mathematics, and perhaps properly appreciate the book.

Most of this went into cold storage for next four years, as I completed a fairly standard degree in math and stat. We did some Fourier analysis, and also some group theory, but didn’t connect the two as Wiener had done. In 1965 I went to teach at Monash University, and began a part-time PhD supervised by Peter Finch. He too was interested in ideas, including information theory, and in due course he got me to teach two undergraduate courses on that topic. I used Wolfowitz’s Coding Theorems of Information Theory, spiced up by Shannon’s classic for the introductory course, and Billingsley’s Ergodic Theory and Information for the advanced course. I loved the ideas, I enjoyed the theory, and it was fun (and quite crazy) trying to teach it to undergrads. I was usually just one lecture ahead of the best students. (My explanation: it was the 1960s.) Also at that time I became exposed to ideas of computability and (un)solvability—themes discussed by Gödel, Turing and others—and learned some Hilbert space theory. Bit by bit, I was getting the background to understanding Wiener’s book.

Fast-forward over 20 years, to when Jorma Rissanen visited Canberra. He talked about MDL and the circle of closely related ideas: the work of Kolmogorov on random numbers, of Chaitin on algorithmic complexity, and of Solomonoff on inductive inference. Statistical theory hadn’t seemed as interesting before, though I never dreamed that I would do research in the area. But soon after meeting Jorma I went to Berkeley, met up with Bin Yu, and they were able to help me contribute a little to the field. So thank you Norbert Wiener for inspiring me with your ideas, and similar thanks to Jorma Rissanen. It was my pleasure to share some of these ideas with Bin Yu, and get an enormous amount back in return.

I am hardly alone in being in love with ideas. But one thing I knew back in 1961, in that library during the breaks between my people watching, was that ideas are just ideas until you have the technical skills to do something with them. At that time I didn’t understand Fourier analysis on groups, the ergodic theorem, linear prediction and so on, and so my appreciation of Wiener’s message was necessarily very superficial. I’m not sure I’m ready even now, as I still know little about psychopathology, Bergsonian time or the firing of neurons. But whether it’s cybernetics or consciousness, evolution or the Big Bang, it is always a long journey to a deeper understanding of the fascinating ideas in science. If we take part of that journey with a brilliant person like Bin Yu, we are extraordinarily lucky. Thanks Bin, and congratulations!