Sequential decision-making in dynamic, uncertain, and adversarial environments is a fundamental challenge in many applications such as online resource allocation, planning, and scheduling. While modern AI and ML algorithms offer transformative potential, their lack of reliability guarantees and possible misalignment with critical design goals—such as fairness—hinder their deployment in high-stakes societal domains. In this talk, I will present our recent progress in developing reliable AI-augmented decision-making algorithms. Our approach adaptively integrates reliable domain expertise to guide and regularize AI-driven decisions based on real-time conditions, exploiting the power of AI for performance improvement while providing reliability guarantees. I will conclude by highlighting applications of our design to problems of societal importance, including health-informed energy management for cleaner air and healthier communities.