This paper proposes a framework that integrates ethics into the requirements elicitation process to build ethically aligned systems. To address the challenges of manual ethical requirements elicitation, which requires gathering diverse input from multiple stakeholders, we present a framework for generating draft ethical requirements by introducing an ethical advocate agent in a multi-agent LLM environment. This agent provides critiques and comments on ethical issues based on system descriptions. We evaluate the proposed framework through two case studies, demonstrating that it captures most of the ethical requirements identified by researchers in a 30-minute interview and suggests several additional relevant requirements. However, it highlights reliability issues in ethical requirements generation, highlighting the need for human feedback in this sensitive area.