Revolutionising our energy systems is indispensable to achieve climate neutrality. Artificial Intelligence (AI) offers great potential for a successful energy transition and is increasingly deployed to enhance energy efficiency, for example by predicting energy demand in demand-side management. While these technologies offer substantial potential to reduce emissions, concerns are emerging about unintended consequences, e.g., their demand for large amounts of energy and how they will affect job security and energy poverty. From an applied psychology perspective, understanding how people perceive, adopt, and interact with such AI-driven technologies is central to whether these solutions succeed at a just energy transition at scale. Crucially, social implications such as trust in or acceptance of AI are rarely assessed. In a previous study on another innovation in the energy sector - energy communities - we highlighted how claims of social benefit often rest on limited evidence and unclear constructs, which underlines the need for systematic evaluation of social impacts (Bielig et al., 2022). Building on this insight, we argue that a similar lack of systematic impact evaluation currently hampers the responsible deployment of AI and risks undermining its contribution to a just energy transition. This contribution addresses this gap by developing a multidimensional impact evaluation framework designed for AI applications in the energy sector. Based on a systematic literature review and expert interviews (N=20), the framework integrates environmental, social, economic, and technical impact dimensions. It provides a modular structure of indicators that practitioners can select according to their specific use case while ensuring comparability. We discuss how our framework supports evidence-based evaluation of AI-based energy solutions, ensuring their effectiveness, efficiency, and fairness.