At the InfoQ Dev Summit in Boston, Justin Sheehy of Akamai delivered an insightful opening keynote on being a responsible developer in the age of artificial intelligence (AI) hype. The talk was aimed at software practitioners who might be feeling overwhelmed by the rapid developments and inflated expectations surrounding AI.
Sheehy began his address by reminding developers that their choices matter significantly in this era. He emphasized three scenarios where developers need to exercise caution: when learning about an AI system, using it wisely and safely, and creating one while minimizing harm.
There are two main takeaways from my talk that I'd like people to have. The first is a call to use their ability as a developer to thoughtfully evaluate the claims about systems that they see, in order to decide in an evidence-based (instead of hype-based) way how best to make use of them. The second is to take responsibility for their own place in these systems, by personally being helpful, honest, and harmless to people through their work. - Justin Sheehy
The speaker stressed that developers have power; they make decisions that can shape technology's trajectory. However, with great power comes great responsibility – a sentiment echoed by tech industry analyst Steve O'Grady over ten years ago.
Sheehy then described what he called "the age of AI," highlighting how impressive strides in AI have been overshadowed by even more massive hype around it. He urged attendees not to fall for grandiose claims about current language models like GPT-4o or Gemini 1.5 from Google without substantial evidence backing them up. He further noted how the makers of LLMs are very quick to point out their deficiencies regarding their own legal liabilities.
He also debunked several misconceptions about these models, such as their ability to develop human-like intelligence or pass Turing tests based solely on convincingly imitating human text. According to Sheehy, these are probabilistic repeating machines akin to parrots mimicking sounds without any understanding or meaning behind those sounds.
The speaker warned against 'bias laundering,' a concept popularized by Margaret Mitchell, where people tend to view answers generated via algorithms as objective facts without considering potential biases ingrained during training stages. This could lead companies into legal trouble if they unknowingly violate safety regulations or rights due to biased outputs from their systems.
Sheehy further cautioned against 'AI washing,' where companies claim their products are powered by advanced AI technologies primarily for marketing purposes rather than actual functionality improvements. Such practices could divert resources from other important work and lead to dangerously wrong decisions based on overconfidence in the AI's capabilities.
The speaker concluded his talk by introducing a framework for 'aligned AI,' which he believes can be applied to developers themselves. This framework's "three H's" are helpful, honest, and harmless. By adhering to these principles, developers can ensure they create technology that truly benefits society while minimizing potential harm.
When asked about a paper at Neurips discussing emergent abilities in LLMs potentially being a mirage, Sheehy noted:
There are a couple of great takeaways from it, such as 'alleged emergent abilities evaporate with different metrics or with better statistics' and 'emergent abilities may be creations of the researcher’s choices' and this paper does a thorough job of showing that these emergent properties emerge more gradually than people may assume depending on how they measure... I would also note that there is another class of alleged emergent abilities (often those that are more dramatically indicative of AGI) that are purely imaginary -- they don't exist at all -- but are often discussed in popular culture. I would advise that people beware of confusing these two.
Developers interested in learning more about the InfoQ Dev Summit series can visit the website and stay tuned to InfoQ for the release of the talk recording in the coming months.