Nowadays, more than 75% of Internet traffic is multimedia traffic, moreover mobile traffic is growing at a rate of 50% each year. All these data together with the evolution of the cloud infrastructures lead us to develop Cloud Mobile Media (CMM) architectures to support the needs demanded by end users. Nevertheless, due to an inherit higher and variable end to end delay mainly as a result of the virtualization process, new challenges appear in particular for live video streaming applications in order to keep a good Quality of Experience (QoE) of the delivered video. Thus, to keep client’s satisfaction within good levels in terms of Mean Opinion Score (MOS), we propose an adaptive QoE-based architecture running on CMM infrastructures for live streaming services. In order to carry out this goal, we propose an estimation of MOS values using an statistical method based on factor analysis. This estimation is based on different measured variables throughout the CMM infrastructure. In addition, we compare the accuracy of the estimated MOS against well-known publicly available video quality algorithms. With these estimations, our proposal is based on two added controllers to the CMM infrastructure: (a) the Software Defined Network controller that acts as a master and (b) the Media Streamer controller. Each one does different actions on the CMM infrastructure in order to maintain and improve the QoE at each end user. Finally, this architecture has been implemented over a fat tree topology in order to show their functionality. The results show that our proposal works properly and it adapts quickly to the network changes in order to deliver a good MOS.