Geometric mechanics is a mathematical discipline that aims to tie various aspects of classical and quantum mechanics together using differential geometry. Beyond the profound beauty of the theory, it has revolutionsed the way we think about physics. Two important processes however, are not treated traditionally in the framework. These are, noise and dissipation. Indeed, real world dynamical systems such as the weather are inherently noisy and dissipative, therefore having a geometric framework for these processes may potentially bring benefit to our further understanding and modelling of these systems. The purpose of this dissertation is to develop a geometric framework for the inclusion of structure-preserving noise and dissipation into mechanical systems and study their implications in the areas of stochastic fluid dynamics and statistical mechanics. In particular, we study the effect of noise added using this framework on the solution behaviours of certain fluid systems, such as the inviscid Burgers’ equation and transport equation of tensor fields. While the noise does not prevent the formation of singularities in the Burgers’ system, it is shown to restore uniqueness of solutions in the tensor transport equation driven by non-smooth drift. We then proceed to add a type of dissipation that balances the energy influx from the noise, resulting in the preservation of Gibbs measure on phase space. This property is then exploited to construct a novel irreversible MCMC algorithm to sample from Lie groups, and also to study phase transition behaviours in a network model consisting of coupled LiePoisson systems with noise and dissipation.