Dielectric capacitors have attracted ever-increasing interest in recent decades for numerous applications in modern electronic and electrical power systems due to their fast charge/discharge speed and high energy density. Novel dielectric materials are highly sought for these capacitive applications. Polymer dielectrics are attractive as they can offer high dielectric strength, low dielectric loss, and light weight, however, a few challenges still exist. For examples, the state-of-the-art polymer dielectric, biaxially oriented polypropylene (BOPP), has low dielectric permittivity, while polyvinylidene fluoride (PVDF) has high dielectric loss. These hurdles require developing next-generation polymers as dielectric materials with new chemistry and unique architectures that are tunable in composition, flexible in mechanical strength and stable at high temperature. In this short review, we begin with some theoretical considerations for the rational design of dielectric polymers with high performance. In the guidance of these theoretical considerations, we review recent progress toward dielectric polymers based on two major approaches, in terms of macromolecular architectures, namely main-chain and side-chain dielectric polymers, and various chemistry and compositions are discussed within each approach.