Inflationary universe has proven to be the most successful way out of the well-known Big-Bang puzzles. Though inflation can naturally be realized in terms of the scalar fields, motivated by the recent anomalies on the CMB data, there could be some hints of having non zero spin fields during inflation. During this talk, we will focus on the gauge field realization of the higher spins as the simplest generalization beyond the scalar fields. We consider this problem in both the theoretical as well as the observational sides. From the theoretical point of view, we first construct a consistent ghost free action including the vector fields in a Bianchi one universe. We will then address how one may generalize the in-in as well as the delta N formalisms in this system. On the observational side, we first explain how this extra field may describe the CMB anomalies and then we go further and would talk about some unique predictions of the primordial anisotropies on the CMB, in the form of the non-zero TB and EB polarizations, as well as the clustering fossils, with a unique distinguishable shape, on the large scale structure.