The geometry and material properties of a room dictate its acoustic behavior, which in turn heavily influences the perception of sounds within. Room acoustics models have thus seen use in a broad range of application areas including architecture and interior design, video games, as well as virtual and augmented reality. With recent advances in the development of machine learning, automatic differentiation frameworks have appeared. These frameworks open a promising avenue for research in acoustical signal processing. We investigate the feasibility of using differentiable physics in determining room shape and material to achieve desired acoustical listening characteristics. We present a simple fully differentiable room acoustics simulator and explore its use for gradient-based optimization of room geometry and material properties in proof-of-concept scenarios. Results and applications in various design and inverse problem scenarios are promising.