Flexible electronics has significantly advanced over the last few years, as devices and circuits from nanoscale structures to printed thin films have started to appear. Simultaneously, the demand for high-performance electronics has also increased because flexible and compact integrated circuits are needed to obtain fully flexible electronic systems. It is challenging to obtain flexible and compact integrated circuits as the silicon based CMOS electronics, which is currently the industry standard for high-performance, is planar and the brittle nature of silicon makes bendability difficult. For this reason, the ultra-thin chips from silicon is gaining interest. This review provides an in-depth analysis of various approaches for obtaining ultra-thin chips from rigid silicon wafer. The comprehensive study presented here includes analysis of ultra-thin chips properties such as the electrical, thermal, optical and mechanical properties, stress modelling, and packaging techniques. The underpinning advances in areas such as sensing, computing, data storage, and energy have been discussed along with several emerging applications (e.g., wearable systems, m-Health, smart cities and Internet of Things etc.) they will enable. This paper is targeted to the readers working in the field of integrated circuits on thin and bendable silicon; but it can be of broad interest to everyone working in the field of flexible electronics.