Mathematics

During the recent Big Data Workshop, Mike Mahoney? mentioned Fast Food (see Revisiting the Nystrom Method for Improved Large-Scale Machine Learning, all the slides of the workshop are here) and it remiuded several items. First to underst...
During the recent Big Data Workshop, Mike Mahoney? mentioned Fast Food (see Revisiting the Nystrom Method for Improved Large-Scale Machine Learning, all the slides of the workshop are here) and it remiuded several items. First to understand Fast Food, you need to understand Random Kitchen Sinks, for that, either you can go back to this 2007 Nuit Blanche blog entry ( Compressed Sensing: Random Features for Large-Scale Kernel Machines ) or just watch the very illuminating explanation by none other than Ali Rahimi: [ The video is from Microsoft Research ( http://research.microsoft.com/apps/video/default.aspx?id=103390 ) . Slides are availabe from that address. several video formats are also available from Microsoft Research ]. Here is the abstract of the talk: Random Kitchen Sinks A popular trend in computer vision, graphics, and machine learning is to replace sophisticated statistical models with simpler generic ones, and to compensate for the missing domain knowledge with huge datasets. These huge datasets in turn require us to solve huge numerical optimization problems that tax popular off-the-shelf implementations of popular algorithms. I describe a randomized way to solve these large scale optimization problems quickly, in a few lines of code, and with provably good performance guarantees. For example, a randomized version of Adaboost and of kernelized Support Vector Machine can fit millions of data points within a few minutes with almost no loss in classification accuracy. Similarly, very large Semi-Definite Programs can be solved quickly by approximating them with suitably randomized linear programs. All of these tricks randomize over most of the variables of optimization and carry out a much cheaper optimization over the remaining variables. A theoretical analysis of these tricks relies on the concentration of measure phenomenon in Banach spaces, and guarantees that in the cases I describe, these tricks work almost as well as carrying out the full optimization. I'll also describe a curious and related way of constructing numerical algorithms that execute reliably on unreliable CPUs that run at subthreshold voltages. The presentation is very well done and I particularly like the connection to hardware at the very end. Alex Smola also provides an explanation on his blog ( The Neal Kernel and Random Kitchen Sinks). So what is Fast Food ? As shown in the recent video ( Fast Food: Approximating Kernel Expansion in Loglinear Time ), Alex thinks that the randomization by the multiplication with a gaussian matrix takes too much time and finds a different structured matrix that resemble a Gaussia to speed up the matrix-vector multiply of the Kernel operation. Think of it as a structured randomization that takes advantage of some speedy matrix-vector multiplication (here the Hadamard transform). In compressive sensing, we have seen those in particular because sometimes the RAM could not hold the measurement matrix. Some have taken advantage of it though the use of Hadamard, some have looked at random Kronecker products. Yes, Virginia, it still is wide open.... Join the CompressiveSensing subreddit or the Google+ Community and post there ! Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.
29 minutes ago
The book A History of Merton College, by G. H. Martin and J. R. L. Highfield, was published by Oxford University Press in 1997. I have had a copy for some time but, to my shame, have only just read it. Roger Highfield was a senior collea...
The book A History of Merton College, by G. H. Martin and J. R. L. Highfield, was published by Oxford University Press in 1997. I have had a copy for some time but, to my shame, have only just read it. Roger Highfield was a senior colleague of mine at Merton. The book is a feast of learning. Here are a few gleanings. The words “university” and “college” could conceivably have interchanged meanings. Martin and Highfield say, A college is essentially and simply an association, a group of people joined together for a particular purpose. The word university originally had similar connotations, and it is not beyond imagination that either could have taken on the other’s meaning. As it happens, and following the example set by Walter de Merton more than seven hundred years ago, colleges have come to be seen as constituents of universities, rather than the other way about. Largely for that reason they are commonly seen as embodied in the buildings which they occupy. Curiously, the colleges in the University of London are now taking on the title and function of universities. It is hard to imagine now the impact that Merton College had at the time of its foundation (the second half of the thirteenth century). Balliol and University Colleges are a few years older, but they were little more than boarding-houses for students; Merton had statutes, a Warden (Peter of Abingdon), a Visitor (the Archbishop of Canterbury), and, crucially, a substantial endowment. The founder intended it to be governed by its scholars, who would also have to take on the substantial task of administering its estates (and fighting legal battles over them where necessary). The statutes of Merton College were copied or adapted for several later foundations including Exeter and Queen’s. The mediaeval dining hall, the first substantial building erected on the site, had the same proportions as the rebuilt modern hall, but must have dominated its surroundings to a much greater extent. And this at a time when the University had no premises of its own. The College had an exchequer chamber which was furnished with a chequerboard table used as an abacus for doing the accounts; indeed, this is the origin of the word “exchequer”. The mediaeval trivium consisted of grammar, dialectic, and rhetoric, as is well known. The use of rhetoric to an aspiring politician or churchman is obvious. “Dialectic” meant, roughly, “logic”, and was also a practical subject: … dialectic could inform the techniques of administration, as it still does in the guise of critical path analysis. Why grammar? There are tantalising hints at its importance: As a conservative theologian [Kilwardby] moved in the following year to condemn some Thomist opinions current in the faculty of arts which he believed would over-heat theological debate and corrupt the faith. The positions condemned seem to have been manifestations of modistic thought, so called from expositions beginning De modis significandi, “the ways of signifying meaning”. Modism sought to relate the classical rules of grammar to Aristotelian philosophy, which Aquinas had striven to accommodate in Christian doctrine. The movement helped to establish the intensive study of grammar as a discipline of its own, rather than as a mere preliminary to the effective uses of languages. The Merton mathematicians are rightly famous. The names of Thomas Bradwardine, William Heytesbury and Richard Swineshead are sufficiently familiar that they could be used as answers in a fairly recent crossword in the Oxford alumni magazine. But there were many more, Thomas Wilton, Walter Burley, John Maudith, John Ashenden, Thomas Buckingham, John Dumbleton, John Tewkesbury, Richard Billingham and William Rede among them. [I did find myself wishing that Martin and Highfield had said more about what they actually did.] So strong was the tradition that several mor
about 3 hours ago
(Click on the cartoon to see the full image.) (C)Copyright 2013, C. Burke.FYI: Caprona is the name of that island (although the book title is just Land) where dinosaurs live in the caves and burrows.
(Click on the cartoon to see the full image.) (C)Copyright 2013, C. Burke.FYI: Caprona is the name of that island (although the book title is just Land) where dinosaurs live in the caves and burrows.
about 6 hours ago
In Slate today, I have a piece about Yitang Zhang’s amazing proof of the bounded gaps conjecture.  Actually, very little of the article is about Zhang himself or his proof; I wanted instead to explain why mathematicians believed th...
In Slate today, I have a piece about Yitang Zhang’s amazing proof of the bounded gaps conjecture.  Actually, very little of the article is about Zhang himself or his proof; I wanted instead to explain why mathematicians believed that bounded gaps (or twin primes) was true in the first place, via Cramér’s heuristic that primes behave like random numbers. And a lot of twin primes is exactly what number theorists expect to find no matter how big the numbers get—not because we think there’s a deep, miraculous structure hidden in the primes, but precisely because we don’t think so. We expect the primes to be tossed around at random like dirt. If the twin primes conjecture were false, that would be a miracle, requiring that some hitherto unknown force be pushing the primes apart.
about 9 hours ago
(a symbol) We can repudiate completely and which we can abandon without regret because one does not know what this pretended sign signifies nor what sense one ought to attribute to it. Cauchy in 1847 in regard to the square root of nega...
(a symbol) We can repudiate completely and which we can abandon without regret because one does not know what this pretended sign signifies nor what sense one ought to attribute to it. Cauchy in 1847 in regard to the square root of negative one..The 143rd day of the year; there are 143 three-digit primes. Also, 1432 is a divisor of 143143. EVENTS1221 "On the first day of the fifth month (May 23), at noon, the Sun was eclipsed and it was total. All the stars were therefore seen. A short while later the brightness returned. At that time we were on the southern bank of the river. The eclipse (began) at the south-west and (the Sun) reappeared from the north-east. At that place it is cool in the morning and warm in the evening; there are many yellow flowers among the grass. The river flows to the north-east. On both banks there are many tall willows. The Mongols use them to make their tents. [Later] (Ch'ang-ch'un) asked (an astronomer) about the solar eclipse on the first day of the month(May 23). The man replied: 'Here the Sun was eclipsed up to 7 fen (6/10) at the hour of ch'en (7-9 h)'. The Master continued, 'When we were by the Lu-chu Ho (Kerulen River), during the hour wu (11-13 h) the Sun was seen totally eclipsed and also south-west of Chin-shan the people there said that the eclipse occurred at the hour szu (9-11 h) and reached 7 fen. At each of these three places it was seen differently. According to the commentary on the ch'un-ch'iu by K'ung Ying-ta, when the body (of the Moon) covers the Sun, then there will be a solar eclipse. Now I presume that we must have been directly beneath it; hence we observed the eclipse to be total. On the other hand, those people on the sides (of the shadow) were further away and hence (their view) gradually became different. This is similar to screening a lamp with a fan. In the shadow of the fan there is no light or brightness. Further away from the sides (of the fan) then the light of the lamp gradually becomes greater." Refers to a total solar eclipse of 23 May 1221. From: Ch'ang-ch'un Chen-jen Tao-ts'ang('The Journey of the Adept Ch'ang-ch'un to the West'). *NASA Eclipse Calendar 1576 Brahe is given use of the island of Hveen for an observatory. [Wadsworth] *VFR1771 Benjamin Franklin visits Joseph Priestley at his home in Leeds just after he begins experimenting with placing mint under a glass to see how long it took to die. Priestly had put insects, small animals, candles etc under glass to measure the time it took to use up the "life force" in the air. To his surprise, the mint flourished in his pneumatic trough. Eventually he would realize that the "spent" air could be rejuvenated by placing a living plant inside the glass. *Steven Johnson, The Invention of Air 1785, a letter from Benjamin Franklin documented his invention of his new bifocal glasses. He was writing from France to a friend describing the solution to carrying around two pairs of glasses to see objects at different distances, with the comment that "I have only to move my eyes up and down as I want to see far or near." Franklin incorporated a two part lens for each eye, each parts having a different focussing power. The invention had limited acceptance at a time when even ordinary spectacles in the colonies already cost as much as $100 per pair. *TIS1825, the electromagnet in a practical form was first exhibited by its inventor, William Sturgeon, on the occasion of reading a paper, recorded in the Transactions of the Society of Arts for 1825 (Vol xliii, p.38). The publication showed pictures of his set of improved apparatus for electromagnetic experiments, including two electromagnets, one of horse-shoe shape and one a straight bar. The formed was bent from a rod about 1 foot (30 cm) long and one-half inch (1.3 cm) in diameter, varnished for insulation, then coiled with a single spiral of 18 turns of stout copper wire. In return for the Society's medal and premium, Sturgeon deposited the apparatus in the museum o
about 11 hours ago
Algebra lessons with detailed examples and videos designed and taught by a teacher.
Algebra lessons with detailed examples and videos designed and taught by a teacher.
about 16 hours ago
An affordable online Algebra Class for students who are struggling with Algebra 1
An affordable online Algebra Class for students who are struggling with Algebra 1
about 16 hours ago
Hypothetical conscious entities that randomly pop into existence may undermine our view of the universe – string theory may banish these Boltzmann brains
Hypothetical conscious entities that randomly pop into existence may undermine our view of the universe – string theory may banish these Boltzmann brains
about 17 hours ago
I covered it last week in A day at the Big Data: Theoretical and Practical Challenges Workshop and Another day at the Big Data: Theoretical and Practical Challenges Workshop but we now have the slides of the Big data: theoretical and ...
I covered it last week in A day at the Big Data: Theoretical and Practical Challenges Workshop and Another day at the Big Data: Theoretical and Practical Challenges Workshop but we now have the slides of the Big data: theoretical and practical challenges workshop organized by Francis Bach and Michael Jordan. Here they are: May 148h30 - 9h10 : Registration and coffee 9h10 - 9h20 : Introduction 9h20 - 10h10 : Chris Holmes, Oxford University?, Bayesian Hidden Markov models with linear time decoding for the analysis of cancer genomes10h10 - 10h50 : Coffee break 10h50 - 11h40 : Eric Moulines, Telecom Paristech, ?Islands Particle model11h40 - 12h30 : Sonia Petrone, Università Bocconi?, Restricted random partitions for Bayesian curve fitting12h30 - 14h : Lunch Buffet 14h - 14h50 : Michael Jordan, U.C. Berkeley?, MAD-Bayes: MAP-based asymptotic derivations from Bayes14h50 - 15h40: Alexandre d'Aspremont, CNRS - Ecole Polytechnique?, Approximation Bounds for Sparse Principal Component Analysis15h40 - 16h20: Coffee break 16h20 - 17h10 : Alfred Hero, University of Michigan, ?Correlation mining17h10 - 18h: Martin Wainwright, U.C. Berkeley, ?Computation meets Statistics: Fast global convergence for high-dimensional (non-convex) statistical recovery May 159h10 - 10h : Leon Bottou, Microsoft Research, ?Large-Scale Learning Revisited?10h - 10h40 : Coffee break 10h40 - 11h30 : Francis Bach, INRIA - ENS?, Stochastic gradient methods for large-scale machine learning11h30 - 12h20 : Ion Stoica, U.C. Berkeley, ?Computations with Bounded Errors and Bounded Response Times on Very Large Data12h20 - 14h : Lunch (take-out) 14h - 14h50 : Piotr Indyk, ?MIT, Faster Algorithms for the Sparse Fourier Transform?14h50 - 15h40: Slav Petrov, Google, ?Large-Scale Language Learning? 15h40 - 16h20: Coffee break 16h20 - 17h10 : Lester Mackey, Stanford University?, Divide-and-Conquer Matrix Factorization17h10 - 18h: Michael Mahoney, Stanford University?, Revisiting the Nystrom Method for Improved Large-Scale Machine Learning18h - 18h20: Conclusion Credit NASA, Opportunity :: Front Hazcam :: Sol 3314, Back to Front Hazcam Sol 3314 Join the CompressiveSensing subreddit or the Google+ Community and post there ! Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.
about 17 hours ago
Guest post by Bruce Bartlett On Monday, David Corfield and Kobi Kremnitzer gave philosophy talks in a snazzy new building at Oxford: Kobi Kremnitzer, What is geometry?, 2-4pm. David Corfield, What might philosophy make of homotopy t...
Guest post by Bruce Bartlett On Monday, David Corfield and Kobi Kremnitzer gave philosophy talks in a snazzy new building at Oxford: Kobi Kremnitzer, What is geometry?, 2-4pm. David Corfield, What might philosophy make of homotopy type theory?, 4.30-6.30pm. The talks shared homotopy type theory as a common theme. The name “Per Martin-L?f” was mentioned a lot, which was good for me since I had always thought Martin and L?f were two separate people: Notes are available above, but I will try to give some brief impressions. Kobi Kremnitzer, What is geometry? 1. Introduction He started by answering the question “Why am I giving this talk?”, and explained that he followed the pragmatic approach to philosophy of mathematics. I think then he said his approach was somehow similar to that of Wittgenstein and Carnap (but he could have been saying the exact opposite :-)), and that for him, there is no Metaphysics in the joint carving. I’m afraid this totally went over my head, but I did imagine some Oxford dons pleasantly carving a roast chicken, which started to make me hungry! He stressed that for him mathematics is not in the business of theorem proving only, but mathematicians create new systems, new languages, and that in the correct language a problem becomes trivial… the approach of Grothendieck. 2. Categorical language Since it was a philosophy talk, he motivated categories by starting with Kripke semantics, going via posets, and then he went from posets to categories by literally “raising the bar”! 3. Crashcourse in algebraic geometry Algebraic geometry is the study of solutions to polynomial equations, like a parabola: X={y−x 2=0}. I haven’t specified what “y” and “x” actually are, and that’s the point. We can interpret them in any ring. Hence the Grothendieck view is to think of an affine variety X defined by a bunch of polynomial equations f 1,…,f n as being a presheaf on the category of rings, X:Rings op→Set, defined by X(R)={(a 1,…,a n)∈R n:f i(a 1,…,a n)=0foralli∈I}. This leads us to define the category of algebraic spaces as being nothing but the category of presheaves on Ring. This category of algebraic spaces has lots of nice properties. Inside it live subcategories of objects having nice properties, such as schemes and sheaves. But Kobi stressed that it is very handy to understand them as living in this general universe of algebraic spaces. 3. What is algebra? There was a crucial idea lurking above - that a ring is a gismo R which allows you to take any polynomial f∈ℤ[x 1,…,x n] and evaluate it on elements of R. So - to go from algebraic to differential geometry, we could replace the concept of a “ring” with the concept of a “C ∞-ring” — this is a gismo R which allows you to take any smooth function f∈C ∞(ℝ n,ℝ) and evaluate it on elements of R! For instance, the space of smooth functions on a manifold is a C ∞-ring… but so is ℂ[ϵ]/ϵ 2 ! So by this slight change of view, we have accomplished Leibniz’s dream - calculus and infinitesimals in the same universe. 4. Final comments He spoke a bit about: derived geometry, set-theoretic foundations, noncommutative geometry, synthetic differential geometry, elementary theory of the category of sets - have a look at the last few pages of his notes. He stressed that ordinary set-theoretic foundations pulverizes spaces into “atomic dust” where the elements have no “cohesion” with each other… we have to put this back in by hand using topology. As a foundation, homotopy type theory will have this cohesiveness natively built in, and that is attractive to a geometer. David Corfield, What might philosophy make of homotopy type theory? Da
about 18 hours ago