Prompt Iteration and Evaluation

Prompt Iteration

Artificial intelligence (AI) is being used by professionals in their daily workplace activities to help generate reports, summarize information and draft emails, among other tasks. When these tools are utilized effectively, they have the potential to greatly increase a worker's productivity. Nevertheless, the quality of the output produced by AI models can depend on the user providing clear and detailed descriptions of what he or she wants produced. As Al Naqbi, Bahroun, and Ahmed (2024) stated, the use of generative AI will increase workplace productivity if the user provides the AI model with a description of what is desired including all necessary context; this will allow the AI model to produce a product that is relevant to the user and of use. This assignment will illustrate how one can create and revise prompts to improve the quality of the output of an AI model for a workplace communication assignment.

Initial Prompt

Draft a memo/email to employees regarding the implementation of a remote work policy.

Output A

The AI model created a brief notice informing employees that a new remote work policy would be put into place shortly, and that further information would follow at a later time. The message did not include key details (e.g., start date, expectations, etc.) that are typically included in an employee communication related to a new policy. In addition, the message had a casual tone and was not suitable for business communication.

Revised Prompt

Write an email from an HR manager that includes a clear description of a new remote work policy that begins on April 1. Explain the remote work policy in the body of the email. Include an invitation for employees to contact HR with any questions they may have regarding the new policy.

Output B

The revised prompt resulted in the production of a more complete email that explained the new policy, included the date it would begin, and had a tone that was professional and acceptable for business communication. Nevertheless, the organization of the message was less than ideal and did not have distinct sections to break up the content.

Revised Prompt

Write an official e-mail from HR to all employees informing them of the new remote work policy going into effect on April 1. The e-mail should have an introduction; clearly state what the new policy entails; identify at least two expected behaviors from employees while they are working remotely (i.e., check-in with your team, meet productivity standards); and close the e-mail with encouragement to contact HR if the employee has any questions regarding the policy.

Output C

The revised prompt generated a structured and professional e-mail. The e-mail had an introduction explaining why the policy was implemented; it had a clear description of how remote work will operate; it had at least two expectations for employees (i.e., maintain open lines of communication, meet productivity levels); and it had a supportive closing urging employees to contact HR with any questions regarding the new policy.

Explanations of Improved Prompt Elements

• Revising the prompt provided additional context and identified the e-mail author as an HR manager, and specified when the new policy goes into effect, therefore generating a more applicable response.
• Incorporating information related to the intended audience (employees) made the e-mail more formal and professional.
• Adding clear structural guidelines generated a more organized and readable output.
• Incorporating examples of expected behaviors (i.e., communicating with colleagues, achieving productivity levels) created a more functional response to the actual needs of workplace communications.

The revised prompts illustrate the improvements in utility and clarity that result from revising the initial prompts to be more applicable for generative AI to assist in enhancing workplace productivity. According to Al Naqbi et al. (2024), the ability to effectively utilize generative AI is contingent upon thoughtfully designing prompts to allow the technology to provide greater assistance to enhance workplace productivity.

Workflow Infographic: Prompt Evolution

The changes in the prompt within the multiple iterations are shown in Figure 1 to enhance the quality of the AI-generated workplace communication. The first prompt was not contextual, and it gave an ambiguous answer. A start date and role context were added in the first revision, which made it more understandable. The last revision brought in order and behavioral norms leading to an orderly and professional work.