Persi Diaconis runs a course on the origins and philosophy of probability called 'Probability, Induction and Statistics'. It draws inspiration and content from Edwin Jaynes' book on probability, and lectures cover some of the profound philosophical debates that have raged through the ages on the meaning and interpretation of probability.
In algorithms, most of us use probability unthinkingly in what statisticians would call "a frequentist vein", where the probability of an event can be envisioned as the fraction of times it occurs on average over a large number of trials. The lectures in Diaconis' course (and indeed much of Jaynes' book) explain why even if you believe this view, it is not as simple as it seems, and how the Bayesian perspective (where probabilities can also represent strength of belief) is a consistent alternate view of the world.
Jaynes also pushed the idea that statistical mechanics can be reformulated via Bayesian reasoning methods; this was a powerful (and controversial) idea, and is too complex to discuss further here. If you're interested in reading more, you can start with Jaynes' papers here and here, and his summary of MaxEnt methods here.
Aside: I've started a QuickLinks section for links that might not necessarily merit a full post, but are interesting nonetheless. It's on the left side of the main page, and has its own RSS feed.