In several combinatorial optimization problems arising in cryptography and design theory, the admissible solutions must often satisfy a balancedness constraint, such as being represented by bitstrings with a fixed number of ones. For this reason, several works in the literature tackling these optimization problems with Genetic Algorithms (GA) introduced new balanced crossover operators which ensure that the offspring has the same balancedness characteristics of the parents. However, the use of such operators has never been thoroughly motivated, except for some generic considerations about search space reduction. In this paper, we undertake a rigorous statistical investigation on the effect of balanced and unbalanced crossover operators against three optimization problems from the area of cryptography and coding theory: nonlinear balanced Boolean functions, binary Orthogonal Arrays (OA) and bent functions. In particular, we consider three different balanced crossover operators, two of which have never been published before, and compare their performances with classic one-point crossover. The statistical comparison shows that for the problems of nonlinear balanced Boolean functions and binary OA the use of balanced crossover operators gives GA a definite advantage over one-point crossover. For the case of bent functions, the situation is reversed, with the unbalanced crossover providing the best performances.