A framework for actively controlling radiated, incident, or local personal sound fields is presented. It relies on loudspeakers and microphones either worn by the user or surrounding the user. The framework aims to address tasks such as speech privacy, personal active noise cancellation, and immersive audio presentation with limited amplification/injection of noise or leakage of private speech into the environment. The formulation relies on modeling and simulation of the sound field using a fast multipole accelerated boundary element method, spectral or point mea-surements of the sound field, and regularized optimization of the field created by actively controlled speakers. The use of acoustic simulation enables the utilization of transfer functions associated with a large number of points distributed in space resulting in effective regularization. Radiation cancellation of up to 20 dB was observed in low frequencies below 1 kHz in a numerical experiment using real-world impulse responses of a wearable loudspeaker setup.