Einstein could have included terms in the equations involving four spacetime derivatives, or six spacetime derivatives, or any even number of spacetime derivatives, but he limited himself to second-order differential equations.

Einstein did not exclude terms with higher derivatives for any practical reason, but for an aesthetic reason: They were not needed, so why include them? And it was just this aesthetic judgment that led him to regret that he had ever introduced the cosmological constant.