Are people using GPT4 successfully in real commercial products yet? Is OpenAI’s latest/greatest API a good hire? A recent timely Ask HN discussion hacker news this week poses this question. Go read the whole thing, or here are my meta-take-ways for anyone trying to build with GPT4 right now.
The model can do useful work today
Some interesting feed across a mix of viable usecases: generating marketing copy and sales emails, to “Correcting or filling missing information in structured data” or Correcting or filling missing information in structured data, or data extraction like from websites or documents, or for internally searching for company information.
But… Availability and performance is a challenge
OpenAI’s APIs aren’t always reachable and response times are variable. Do work around managing for retries, or fallback to other processes if OpenAI is not available. Rate limits are a challenge too, and the process for appealing those is going to be difficult with the current level of demand.
Mix GPT4 and 3.5 versions for speed and Cost considerations
GPT4 is the most accurate but also the slowest and more expensive per call. But you can also try mix and match for usecases where 3.5 is good enough or as a first pass. Test and optimize. At MainStreet, we had good success using basic GPT3.5 to generate super-specific customer help and training material. E.g. “draft a help center article on how [general finance concept] might apply to expenses for [specific job role category] in [customers specific business vertical]. We’d generate this kind of content offline, then review it before publishing. Even with manual review, the speedup vs generating a broad set of help content from scratch was enormous.
Keeping human review in the loop or being clear/transparent with your users
Either it seems people are mostly using GPT internally, where quality of output just has to better or more scaleable than a previous process. Or folks are building apps that explicitly expose the AI to their customers, but adding value through a novel UI or domain-specific assistance with prompting. The ‘copilot’ modality when coding or creating is already a proven commercial model and incredibly popular. Will some equivalent work for banking, investing or financial management apps? It will be interesting to see how easily all of this extends into more regulated or professional/fiduciary responsibility domains. But with the right controls, transparency and model-refinement, it will get sorted out.
Key Takeaway: The big cloud vendors will probably make all this better, just be prepared to pay. The business case here is pretty clear for Azure, Google and AWS et all. Offering, enterprise grade availability, as well as data privacy for custom-trained/refined LLM models is going to be huge business. I could also see opportunities major vertical-oriented players offering something similar. Bloomberg has announced their GPT model for Finance. I’d like to see what Stripe or Visa do for models trained on payments, or models for retail banking, lending, accounting, insurance etc.
Relevant links:
Ask HN: Who has deployed commercial features using GPT4?
Open AI API docs
Microsoft Azure OpenAI services
Google Cloud AI (Note Google Bard API is still closed and invite-only)
Amazon AWS Generative AI Announcement (Announced April 13/2022)
Bloomberg GPT (Announced March 30)
image credit: Midjourney “A robot works in an office, ai, paperwork, midcentury modern”