Futapy

HW 6

There have been some fascinating articles that came cross my feed this week:

• https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/open-ais-powerful-text-generating-tool-is-ready-for-business.amp.html
• also https://venturebeat.com/2021/02/27/gpt-3-were-at-the-very-beginning-of-a-new-app-ecosystem/
• But this one is the OMG moment: there are multi-modal neurons in CLIP per https://distill.pub/2021/multimodal-neurons/. "All of these features, even the trivial-seeming ones, have rich multimodality, such as a yellow neuron firing for images of the words “yellow”, “banana” and “lemon,” in addition to the color."
  • The Independent reports a summary

This is an AI experiencing "yellow" and "Spider-man" as a concept, or qualia.

OK on to the homework, per http://informationcomputingandthefuture.blogspot.com/p/hw-6.html

A1a. Project: Research paper. I haven't done one of these in decades so this may be a stretch, but definitely Topics, Trends and Issues in large-scale AI with an emphasis on natural language processing models. Previously I had stated "Research OpenAI and GPT3; get a sense of the scale of what they are doing and how it could profoundly impact out lives in the future, and present a 20 minute slide-deck summary of what I found."

I have a question; what is your expectation of the difference between a slide deck with fully fleshed out text, and a research paper? How formal (MLA styles, citations, disallowed sources) should the research paper be?

A1b.  an outline / skeleton to start flesh out:

• Brief history of AI up to about 2015 and recent developments
  • timeline of major developments
  • NLP and computer vision research and techniques
  • Overview of important contributors to the field
  • Predictions from the past
    • Kurzweil on the singularity
    • Dennett on the mechanics of cognition
    • Sean Dorrance Kelly on why creativity is a human endeavor

• Overview of OpenAI NLP models GPT2, GPT3, others? CLIP?
  • what is the size of the trained model

• Brief history of OpenAI (the company and the products)
  • Intent / purpose?
  • Structural changes that made it possible
    • Amazon/Google scale out architectures
      • kubernetes?
    • Elon Musk and other founder support
  • Technical breakthroughs that made large generative models possible
    • Generative techniques
  • supervised vs. unsupervised training
    • supervised is labor intensive
    • limited curated data sets vs. the whole of the internet
    • overcoming biases in unfiltered sources

• Capabilities and current uses
  • research
  • advertising, social media, etc.
  • fake news
  • Dangers of radicalization

• Cost to train
  • compute time
  • actual dollars
  • equivalent environmental impact / CO2 release

• Cognition, creativity and qualia
  • Multimodal neurons identified in CLIP
    • two of five Human senses; how to add third? all five? more (RF)?
  • Perhaps the Dawkins cognition prediction from Predictions

• Extrapolation to the future
  • Ian from Jurassic Park: "Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should"
  • Danger: when a computer can organize and synthesize information better then a human, what will be the motivation to think?
  • What becomes of art?
  • Incorporate what I've already written on how this might impact individuals, organizations and society as a whole

A1c. Much of the future of this has already been thought; skim reporting from The Singularity is Near and Intelligence Explosion to see if what I am seeing fits into those frameworks, borrow some of their observations and evaluate their predictions

A2: I will discuss electric vehicles (EVs), particularly cars, the production kind, that you can buy from a large auto manufacturer. According to the DOE, electric cars have been around for around 130 years and had a good initial run before being completely displaced in the automobile market by internal combustion engines in the early 20th century when the cost of the Ford Model T and the subsequent invention of the electric starter for gasoline engines. They largely remained the domain of hobbyists until the 2010s or so when battery advances once again made them competitive against cars with internal combustion engines for specialized use cases and emission reasons. In the late 1990s GM produced a limited run of electric cars for lease, the EV1, but ended the project in 1999 and most of the EV1s were recalled and destroyed. Battery technology continued to improve and since roughly 2010 the range and quality of electric cars has improved while the price has stayed somewhat above that of an equivalent traditional car.

• Kline and Rosenberg model: This is a good fit for the history of the electric car; as the market for EVs changed and competitive technologies advanced, EVs have continually changed. EVs, the technology and associated cost to make them, and the market have gone through several cycles of improvement: DC to AC drive, resistive to solid-state drive, and various battery chemistries including lead-acid, large format NiMH, and lithium ion. Large format NiMH in particular was developed largely for EVs so a feedback cycle can be established.
• Abernathy and Utterback model: (fluid, transitional and specific phases) This model doesn't fit as well; at best after 130 years we are still in the fluid phase with new battery and motor formats being tried. An argument could be made we are in the transitional phase; EVs today are largely based on lithium-ion batteries, electronic inverters, and AC induction motors, and Tesla has certainly increased production based on this model. Charging systems have some standards which have allowed public charge points, but use cases continue to evolve and those standards are still undergoing innovation and change. This model might be more appropriate for innovations without technological dependencies, such as the Swingline 747 Stapler.
• Clark and Henderson model (improvements to components / changes in system architecture) Certainly improvements to components are driving the majority of change in EVs currently. Changes in architecture will come about if the world (or our corner of it) becomes serious about climate change and decarbonization, if the economics of electricity vs. fossil fuels are upended, or if radical breakthroughs in electricity storage happen. One could argue that the assembly line of the Model T was the architectural change that popularized inexpensive internal combustion engines over more costly EVs in the early 20th century.
• Teece model (imitation and companies with strong complementary services win): I think the Tesla story upends this model. The giant automakers had all the complementary services: decades of experience, deep supply chains, deep pockets, dealer networks, advertising departments - and still Tesla was able to create a better, more attractive product and dominate the consumer EV market. Perhaps this model fit the Rand, IBM, GM and Westinghouse world but the quicker moving and risk embracing companies like Apple and Tesla are eating their lunch. Perhaps the Teece model fits innovations like the ball bearing that are onto the specific phases of Abernathy and Utterback, but by then all the innovation has already happened.
• Christensen I model (resources, processes, values): This model may be a good fit in that the companies that are successful selling EVs today have values that support an EV as a product rather than just a compliance requirement, and processes designed to make them more cost effective and efficient. The Fiat-500e may have been the best EV made but the CEO of the company publicly said "please don't buy one" all but guaranteeing its failure with his values statement, and certainly the CEO wasn't interested in refining processes at that time. GM is limited in that their dealer network values selling higher profit vehicles like trucks and SUVs and has processes in place (and service departments) that depend on the internal combustion engine. As a values example, we test-drove a 2015 Volt from a dealership who had one example in stock, around back, with a dead battery and a flat tire. While the CEO may be all-in on electric, convincing the dealership and service departments to change their values and processes will take time.
• Christensen II model : (Disruptive vs. Sustaining Innovations) In general EV development in the U.S. since 2010 has been in response to federal and state mandates for fleet emission standards and zero emission vehicle targets as well as tax incentives for new EV purchases. These rules created new market segments that were filled slowly at first with lower-end compliance models from traditional manufacturers, and more rapidly in the last few years by Tesla who has disrupted the high end of the auto market in the U.S. by making their cars more of a high performance status symbol once associated with Audi, BMW, or Mercedes. Both of these could fall into the "Disruptive innovations" category of this model.
• Value Chain Evolution model (Christensen III; integration vs. specialization): this model may not apply; it seems to assume a transitional or stable phase with a high degree of competence in-house and cheaper products available from suppliers. GM took the specialization approach to its Volt and Bolt EV program in that LG Chem is the specialized provider of the battery and electronics because GM clearly didn't have what was good enough in-house, going against this model's "outsource what is better than necessary" philosophy. In the case of the second generation of volt, GM actually integrated some of the expensive parts that were better, and more expensive, than necessary. Admittedly I don't have a deep understanding of EV manufacturing supply chains but even after 130 years I don't think the industry has developed to the point that this model is a good fit.

By the way - we (my wife and I) are EV tourists; we took a ride in (UALR's own) David Luneau's "Voltsrabbit" conversion a couple decades ago, as well as a really nice roadster retrofit made with used Tesla parts up in Quebec just a couple years ago.

A3: Book pg 56-75, biographies and essays by Richard Dawkins and Daniel Dennett.

I had never read Dawkins but was exposed in undergrad to the idea of "memes" or thoughts that reproduce and evolve similar to a life form. Reading his brief biography and overview of his work, I am stuck that I have come to some of the same conclusions that he writes about in The Extended Phenotype, that our genes are more in control of our behavior than we would like to think and much of our behavior is in service of our gene's survival. I might go further to say that racism and xenophobia could be explained by the gene directing us to help those individuals descended from it first, those individuals with genes most like it second, and lastly if at all, those individuals that that don't appear like it (I am not apologizing for racism here - just wondering if it is our gene's directed innate judgement of others based on a subjective judgement of how genetically similar they are to ourselves - and if this is the case, then we have a much larger obstacle to overcome than commonly admitted) I disagree with Dawkins in that he seems to downplay the awe one has when marveling at all of creation, but I agree that we need to get on with the process of understanding it instead of waving away that complexity with our reliance on myth. His prediction, more of a hope, is that in this century we will understand what consciousness is and how it works.

Daniel Dennett is another author I am not familiar with but seems in the same stripe as Dawkins and maybe a bit more militant about his hostility towards religious explanations of the world. His opinion that the source of creativity is completely in the realm of the mechanical is in line with my thinking; in fact, if a feeling or thought can be instantly summoned by poking an area of the human brain (it can), then it is fairly obvious to me that most or all that we are is encoded within the matter within our heads. This has been reinforced twice in my life by watching brilliant, conscious people once full of vitality slowly slip away due to brain disease until nothing but body was left. However Dennett seems overly confident in his opinion that the phenomenon of human consciousness will be explained mechanically; for example, we still have much to learn about particle interactions in hidden dimensions and the marvelous complexity of spacetime that may contribute to the beautiful randomness of creativity. We may be DNA robots, but we're beautiful, enigmatic and potentially unknowable DNA robots. I did write "wow" in the margin on his discussion of truth and how one might not want to unleash it on the world due to misinterpretation of its representation, and how that misrepresentation might become a meme itself. His prediction is that in this century it will be memes, both tonic and toxic, that will wreak havoc on the unprepared world just as European germs did to unprepared natives in the lands that were colonized, and it will be a great challenge to extend public health to cultural health. Wow again.

20 amazing pages. A+; would read again.

A criticism of this book could be that it is comprised of the works of people who are relatively famous; they have largely stepped away from their research and laboratory work to write books for public consumption and have been successful at that. Does that make them "great minds" or simply acknowledge that they are already famous? Is fame the reward of winning in the marketplace of ideas? An agreement with reviewers is it is a great book for finding authors that resonate with ones own thinking. I'm not certain that's a good thing, but I agree.