To enhance the performance of wind farms, the yaw-based wake steering control strategy has emerged as a promising approach and gained increasing attention in the last decade. This study focuses on the classical open-loop implementation of this strategy, which is typically built upon wake models for affordability. The practical robustness of the strategy, with respect to either a static error in the measured absolute wind direction, or a nacelle positioning uncertainty, is assessed. In particular, and while mentioned to be critical in several prior works, the static error in the wind direction is often overlooked. In this work, a real four-turbine offshore layout was simulated under a truly neutral atmospheric boundary layer with the Large Eddy Simulation (LES) coupled code YALES2-BHawC. For the supplementary purposes of comparison and validation, engineering wake models from the FLORIS library were also used to predict the power production, and SCADA data were considered in the baseline case. It was observed that a few-degree static error in the absolute wind direction, which would typically emerge from a calibration bias, is enough in some circumstances to turn a significant expected power benefit into slight losses. Relatively, a typical nacelle position uncertainty only has a marginal impact on the power benefit. When existing, the total power benefit showed to be mainly achieved on the most downstream turbines. Furthermore, despite an overall agreement between the wake models and LES in predicting total power trends, detailed comparisons revealed noteworthy discrepancies related to wake combinations and wake-added turbulence. In particular, it was observed that strong wake overlap conditions result in a significant increase of the meandering amplitude, making it more challenging to get a reliable assessment of the total power benefit on a reasonably short period of time. To ultimately secure a significant power gain, these findings suggest that new efforts remain desirable on the modeling, implementation, and validation sides of open-loop wake steering control.