We present OptiMUS-0.3, a large-scale language model (LLM)-based system designed to formulate and solve (mixed-integer) linear programming problems described in natural language. OptiMUS-0.3 performs functions such as developing mathematical models, writing and debugging solver code, evaluating generated solutions, and improving the efficiency and accuracy of the models and code based on the evaluations. Its modular structure allows it to handle problems with long descriptions and complex data, and we experimentally demonstrate that it outperforms existing state-of-the-art methods by at least 22% on easy datasets and by at least 24% on difficult datasets (including the new dataset NLP4LP released with this paper).