Estimation problems in statistics can often be formulated
as nonlinear optimization problems where the approximating function
is also constrained to satisfy certain shape constraints. For
example, it may be required that the approximating function be
nonnegative, nondecreasing or nonincreasing in some variables,
unimodal in a variable, convex, or concave. We show how these shape
constraints can be included in models where the approximating
function is a polynomial spline of a given degree. Four classes of
problems are considered: univariate nonparametric regression with
nonnegativity constraints; isotonic, convex, and concave
nonparametric regression; density estimation; and univariate
arrival rate approximation. It is shown that all of these problems
can be modeled by convex optimization models with linear and
second-order conic constraints, which can be handled very
efficiently both in theory and in practice. It is shown that in
certain cases a linearly constrained model is enough, while in
other cases nonlinear constraints are unavoidable. Extensive
numerical computations show that these models often outperform
or match the quality of results obtained using other popular
nonparametric statistical methods, such as state-of-the-art kernel
methods.