Prompt Engineering: An Integrated Dream

By neub9
6 Min Read


Prompt Engineering: Myth vs Reality

Image created by me with Microsoft Image Creator

Since OpenAI unleashed ChatGPT to the public, a flurry of discussions has emerged online about a new dream job: Prompt Engineering. It’s touted as “AI’s Hottest Job,” promising six-figure salaries without the need for programming experience. Enthusiasts describe it as a job of the future, where anyone can earn up to $335K by smooth-talking a cool know-it-all robot into giving right answers. No surprise, Instagram money making sages, Youtube career preachers, and self-proclaimed oracles of Tiktok have been very vocal about it. While this sounds like a dream job, is it truly achievable? Let’s delve into the reality of job market behind the hype to find out.

Analyzing job advertisement data provides valuable insights into labor demand trends, responsibilities, qualifications, and salary expectations. Thus, I decided to take a look at the ad data of the so-called “AI’s Hottest Job” with no speculations or presumptions. I collected 73 recently posted unique job ads data from popular on-line job posting platforms. Read about my data collection methodology and access the data set here. While 73 may not be an ideal sample size, it’s a comprehensive starting point for our analysis. The initial revelation is sobering: there’s a scarcity of employers seeking “prompt engineers.”

Now, let’s take a look at the data. The most frequently mentioned job title is “prompt engineer.” However, other titles such as “IT Innovation Analyst,” “Freelance ML/AI Engineer,” “Data Scientist,” and “AI Engineer” also emerge.

Next, I used ChatGPT and Claude to summarize the collected ads text corpus to identify top prompt engineering qualifications and qualifications. I did multiple rounds of prompting with different approaches followed by manually checking the data to make sure I got stable and valid output.

Essential qualifications demanded for Prompt Engineer job:

  • Proficiency in Python programming (2-5 years of experience) including experience with AI/machine learning frameworks like TensorFlow, PyTorch, Keras.
  • Working knowledge of NLP and LLMs (2-5 years of experience) like BERT, GPT-3/4, T5, etc.
  • Strong analytical and problem-solving skills.
  • Expertise in prompt engineering principles and techniques.
  • Excellent communication skills, both verbal and written.

And the essential responsibilities of the prompt engineering jobs are:

  • Prompt Design and Optimization
  • Integration and Deployment
  • Performance Evaluation and Improvement
  • Collaboration and Requirements Gathering
  • Knowledge Sharing

Job ads data analysis of degree requirements indicates a preference for technical educational backgrounds in computer science, math, analytics, engineering, physics, or linguistics. A bachelor’s degree in computer science or a related field is commonly required, with more advanced degrees preferred or required for senior roles. The salaries are very different depending on the responsibilities and seniority. It can be as low as 30k and as high as half a million dollars per year. On average, the job ads with salary information pay between 90k and 195k a year.

Despite initial enthusiasm, doubts regarding the viability of prompt engineering as a dream job have surfaced. As Ethan Mollick, the Wharton School professor, wrote in a twitter post last year “prompt engineer is not a job of future” because “AI gets easier” and smarter in interpreting basic prompts. A month ago Coursera published a well-thought career guide for prompt engineering (also see this). It seems the initial Gen AI fad is slowly fading, and we are in a better position to understand AI’s current status and future trends. However, “prompt engineering” is not (and it never was) a dream job that some people wanted it to be. Without significant experience in programming, natural language processing, machine learning, product development, and software integration no one is going to pay you a six-figure salary for just smooth-talking ChatGPT into a right answer.

The present and future of prompt engineering, and Gen AI applications, seem to be influenced by two important trends: first, as Ethan Mollick mentioned, Gen AI models are getting more adept at generating good outputs from unsophisticated simple prompt, perhaps similar to how Internet search engines have become better at returning more relevant results from simple search queries. Second, Gen AI models are being increasingly integrated into the business’s products, services, and platforms. This adaptation is crucial for the success of the AI economy. Therefore, knowing how to optimize, fine-tune, customize, and integrate Gen AI models with the current information systems and products is and will remain a valuable skill set. That’s why in the current prompt job ads, there is a huge demand for programmers, system designers, and those who can collaborate with other product development team members.

Mahdi Ahmadi is a clinical assistant professor at the Information Technology & Decision Sciences department at the University of North Texas where I teach data mining, business intelligence, and data analytics. My primary research area is the application of machine learning and data mining techniques in businesses. I also provide consultation to businesses, higher education institutions, and non-for-profit organizations on their data analytics problems.


Share This Article
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *