Paulo Macedo: “A retenção de talento é um problema maior do que a regulação”

[MarcoAntonio]

Nova arquitectura que tem estado a dar que falar, com resultados promissores em várias áreas de investigação incluindo NLP (a área a que dizem respeito os LLMs): MAMBA.

O paper original:

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

A arquitectura deste novo modelo substitui o mecanismo de atenção (attention mechanism) no centro da arquitectura dos Transformers por um mecanismo de selecção no contexto de um modelo de espaço de estados (Selective SSM) obtendo resultados equiparáveis em termos de perplexity e de accuracy a modelos baseados em Transformer com o dobro de tamanho. Resultados que são muito promissores, apontando para a maior eficiência (e também menores custos/tempo de treino e de inferência) comparado com os modelos baseados em Transformers que têm dominado nos últimos anos.

A arquitectura ainda não foi testada com tamanhos gigantes tipo os do ChatGPT 3 e 4 mas os resultados com redes mais pequenas são já muito interessantes e apontam no sentido da escalibilidade.

---

Um resumo também aqui na primeira metade deste artigo:

Mamba, the ChatGPT Killer & The Great Deflation

---

Github do Mamba, disponível publicamente:

https://github.com/state-spaces/mamba

[MarcoAntonio]

Este artigo remete para alguns posts meus aqui no tópico durante o ano passado, mas o tema é sempre actual.

LLMs Are Dumber Than a House Cat (Towards Data Science)

(...)

"We're easily fooled by those systems into thinking they are intelligent just because they manipulate language fluently.

The only example that we have of an entity that can manipulate language is other humans so when we see something that can manipulate language flexibly we assume that entity will have the same type of intelligence as humans but it's just not true.

Those systems are incredibly stupid.

Partly they are stupid because they are only trained on language and most of human knowledge has nothing to do with language.

To some extent the smartest AI systems today have less understanding of the physical world than your house cat."

— Yann Lecun, Chief AI Scientist for Facebook AI Research (FAIR).

(...)

(...)

Similar trends have been observed in other studies where developers, data practitioners, and business consultants became twice as fast in certain tasks when using LLMs. Output quality also increased.

You'll see a lot of "Can LLMs replace a Data Scientist?" and "Can LLMs replace Developers?" For now, the answer to these questions is: No. LLMs won't replace you, but people who use LLMs will.

(...)

[perneta]

Eu arriscaria a dizer que se os LAM funcionarem e forem o caminho no que diz respeito ao interface entre utilizadores e dispositivos/aplicações, então serão a seu tempo incorporados nos sistemas operativos basicamente a custo zero, substituindo os "assistentes" actuais...

Em todo o caso, a ideia é interessante mas requere ainda exploração. Concordo com ele que o actual interface dos telemovóveis (por exemplo) com o tempo tornou-se uma confusão e pouco, nada prático. Precisa de ser repensados. Mas o que precisamos mesmo é de repensar a interacção com os dispositivos/aplicações mesmo... e não de acrescentar mais um dispositivo na cadeia que não substitui os restantes.



Ainda assim, sempre é mais interessante que o AI Pin, que me parece não passar de um completo money grab à boleia da popularidade do ChatGPT.

É interessante, mas concordo que se pode ser integrado no smartphone, não faz muito sentido utilizar 2 dispositivo. É claro, não tirando o mérito a teenage engineering que apresenta sempre designs espetaculares.

Percebo a questão da interface, contudo, vejo aquilo mais numa perspectiva de complemento, como assistentes.

[MarcoAntonio]

Eu arriscaria a dizer que se os LAM funcionarem e forem o caminho no que diz respeito ao interface entre utilizadores e dispositivos/aplicações, então serão a seu tempo incorporados nos sistemas operativos basicamente a custo zero, substituindo os "assistentes" actuais...

Em todo o caso, a ideia é interessante mas requere ainda exploração. Concordo com ele que o actual interface dos telemovóveis (por exemplo) com o tempo tornou-se uma confusão e pouco, nada prático. Precisa de ser repensados. Mas o que precisamos mesmo é de repensar a interacção com os dispositivos/aplicações mesmo... e não de acrescentar mais um dispositivo na cadeia que não substitui os restantes.



Ainda assim, sempre é mais interessante que o AI Pin, que me parece não passar de um completo money grab à boleia da popularidade do ChatGPT.

[perneta]

Mais pela ideia e direcção do que pelo dispositivo em si, que duvido fortemente que vá funcionar tão bem quanto ele alega (e se chegar sequer ao mercado). Aliás, se funcionar e se realmente este (os LAM) for o caminho então não irá ser preciso um dispositivo ad hoc, será incorporado nos sistemas operativos dos dispositivos base como smartphones e PCs e pronto.

Está engraçada a ideia! Será mais pratico e económico, ser incorporado nos dispositivos.

[BearManBull]

$199

[MarcoAntonio]

Mais pela ideia e direcção do que pelo dispositivo em si, que duvido fortemente que vá funcionar tão bem quanto ele alega (e se chegar sequer ao mercado). Aliás, se funcionar e se realmente este (os LAM) for o caminho então não irá ser preciso um dispositivo ad hoc, será incorporado nos sistemas operativos dos dispositivos base como smartphones e PCs e pronto.

[MarcoAntonio]

Teaching LLMs To Say, “I don’t know”

Instead of stating that it does not know, LLMs hallucinate. Hallucination can best be described as highly plausible, succinct & reasonable responses from a LLM, but which is factually incorrect.

Introduction

A few years back IBM Watson Assistant introduced functionality to define and detect user requests which falls outside of the chatbot’s domain. I believe this was one of the first forays into the realm of defining what the ambit of the chatbot was.

Chatbots in the early days were plagued by the problem of getting stuck on “Sorry, I can’t help you with that”. And subsequently encouraging the user to rephrase their input without any attempt from the chatbot to facilitate the progression of the conversation.

Fast Forward To LLMs

With Large Language Models (LLMs) we are on the other side of the spectrum, where the Conversational UI always responds with an answer and rarely says, I don’t know.

Compared to traditional chatbot architecture and performance, LLMs mesmerise us with their outstanding ability of fluency, coherence and ability to maintain context. But, LLMs are still likely to hallucinate unfaithful and nonfactual responses.

(...)

O artigo a que ele se está a referir:

R-Tuning: Teaching Large Language Models to Refuse Unknown Questions




Adenda/update

Survey com mais de 30 papers e diferentes métodos de mitigação da alucinação nos LLMs, incluindo o paper de cima:

A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models

[BearManBull]

IA. New York Times processa OpenAI e Microsoft por usarem artigos do jornal para treinar modelos como o ChatGPT

É a nova moda agora querem todos ganhar dinheiro por dados de treino.

A China esfrega as mãos porque ali os NYT deste mundo fecham a boca e estão caladinhos.

[Opcard]

A Albânia vai utilizar a IA para transferir o acervo comunitário para a sua legislação na tentativa de adesão , poupado muitas dezenas de milhões de euros pois teria de pagar ao juristas linguistas a grande dúvida é se a Europa irá aceitar se aceitar, é uma revolução .

São na totalidade mais de 200 000 páginas , só para o chamado direito comunitário derivado 80 000 páginas , e cada ano são acrescentados 2 500 actos legislativos .

Se for aceita todos os outros países a espera de adesão farão o mesmo em média estes profissionais dia ganham 750 euros .

[MarcoAntonio]

E já agora, não me recordo se chegou a passar aqui no tópico...

RT-2: New model translates vision and language into action

[MarcoAntonio]

Google launches Gemini, the AI model it hopes will take down GPT-4

Google has been an ‘AI-first company’ for nearly a decade. Now, a year into the AI era brought on by ChatGPT, it’s finally making a big move.

It’s the beginning of a new era of AI at Google, says CEO Sundar Pichai: the Gemini era. Gemini is Google’s latest large language model, which Pichai first teased at the I/O developer conference in June and is now launching to the public. To hear Pichai and Google DeepMind CEO Demis Hassabis describe it, it’s a huge leap forward in an AI model that will ultimately affect practically all of Google’s products. “One of the powerful things about this moment,” Pichai says, “is you can work on one underlying technology and make it better and it immediately flows across our products.”

Gemini is more than a single AI model. There’s a lighter version called Gemini Nano that is meant to be run natively and offline on Android devices. There’s a beefier version called Gemini Pro that will soon power lots of Google AI services and is the backbone of Bard starting today. And there’s an even more capable model called Gemini Ultra that is the most powerful LLM Google has yet created and seems to be mostly designed for data centers and enterprise applications.

Google is launching the model in a few ways right now: Bard is now powered by Gemini Pro, and Pixel 8 Pro users will get a few new features thanks to Gemini Nano. (Gemini Ultra is coming next year.) Developers and enterprise customers will be able to access Gemini Pro through Google Generative AI Studio or Vertex AI in Google Cloud starting on December 13th. Gemini is only available in English for now, with other languages evidently coming soon. But Pichai says the model will eventually be integrated into Google’s search engine, its ad products, the Chrome browser, and more, all over the world. It is the future of Google, and it’s here not a moment too soon.

OpenAI launched ChatGPT a year and a week ago, and the company and product immediately became the biggest things in AI. Now, Google — the company that created much of the foundational technology behind the current AI boom, that has called itself an “AI-first” organization for nearly a decade, and that was clearly and embarrassingly caught off guard by how good ChatGPT was and how fast OpenAI’s tech has taken over the industry — is finally ready to fight back.

So, let’s just get to the important question, shall we? OpenAI’s GPT-4 versus Google’s Gemini: ready, go. This has very clearly been on Google’s mind for a while. “We’ve done a very thorough analysis of the systems side by side, and the benchmarking,” Hassabis says. Google ran 32 well-established benchmarks comparing the two models, from broad overall tests like the Multi-task Language Understanding benchmark to one that compares two models’ ability to generate Python code. “I think we’re substantially ahead on 30 out of 32” of those benchmarks, Hassabis says, with a bit of a smile on his face. “Some of them are very narrow. Some of them are larger.”

In those benchmarks (which really are mostly very close) Gemini’s clearest advantage comes from its ability to understand and interact with video and audio. This is very much by design: multimodality has been part of the Gemini plan from the beginning. Google hasn’t trained separate models for images and voice, the way OpenAI created DALL-E and Whisper; it built one multisensory model from the beginning. “We’ve always been interested in very, very general systems,” Hassabis says. He’s especially interested in how to mix all of those modes — to collect as much data as possible from any number of inputs and senses and then give responses with just as much variety.

Right now, Gemini’s most basic models are text in and text out, but more powerful models like Gemini Ultra can work with images, video, and audio. And “it’s going to get even more general than that,” Hassabis says. “There’s still things like action, and touch — more like robotics-type things.” Over time, he says, Gemini will get more senses, become more aware, and become more accurate and grounded in the process. “These models just sort of understand better about the world around them.” These models still hallucinate, of course, and they still have biases and other problems. But the more they know, Hassabis says, the better they’ll get.

Benchmarks are just benchmarks, though, and ultimately, the true test of Gemini’s capability will come from everyday users who want to use it to brainstorm ideas, look up information, write code, and much more. Google seems to see coding in particular as a killer app for Gemini; it uses a new code-generating system called AlphaCode 2 that it says performs better than 85 percent of coding competition participants, up from 50 percent for the original AlphaCode. But Pichai says that users will notice an improvement in just about everything the model touches.

Equally important to Google is that Gemini is apparently a far more efficient model. It was trained on Google’s own Tensor Processing Units and is both faster and cheaper to run than Google’s previous models like PaLM. Alongside the new model, Google is also launching a new version of its TPU system, the TPU v5p, a computing system designed for use in data centers for training and running large-scale models.

Talking to Pichai and Hassabis, it’s clear that they see the Gemini launch both as the beginning of a larger project and as a step change in itself. Gemini is the model Google has been waiting for, the one it has been building toward for years, maybe even the one it should have had ready before OpenAI and ChatGPT took over the world.

Google, which declared a “code red” after ChatGPT’s launch and has been perceived to be playing catch-up ever since, seems to be still trying to hold fast to its “bold and responsible” mantra. Hassabis and Pichai both say they’re not willing to move too fast just to keep up, especially as we get closer to the ultimate AI dream: artificial general intelligence, the term for an AI that is self-improving, smarter than humans, and poised to change the world. “As we approach AGI, things are going to be different,” Hassabis says. “It’s kind of an active technology, so I think we have to approach that cautiously. Cautiously, but optimistically.”

Google says it has worked hard to ensure Gemini’s safety and responsibility, both through internal and external testing and red-teaming. Pichai points out that ensuring data security and reliability is particularly important for enterprise-first products, which is where most generative AI makes its money. But Hassabis acknowledges that one of the risks of launching a state-of-the-art AI system is that it will have issues and attack vectors no one could have predicted. “That’s why you have to release things,” he says, “to see and learn.” Google is taking the Ultra release particularly slowly; Hassabis compares it to a controlled beta, with a “safer experimentation zone” for Google’s most capable and unrestrained model. Basically, if there’s a marriage-ruining alternate personality inside Gemini, Google is trying to find it before you do.

For years, Pichai and other Google executives have waxed poetic about the potential for AI. Pichai himself has said more than once that AI will be more transformative to humanity than fire or electricity. In this first generation, the Gemini model may not change the world. Best-case scenario, it might just help Google catch up to OpenAI in the race to build great generative AI. (Worst-case scenario, Bard stays boring and mediocre, and ChatGPT keeps winning.) But Pichai, Hassabis, and everyone else at Google seem to think this is the beginning of something truly huge. The web made Google a tech giant; Gemini could be even bigger.

[BearManBull]

Gemini is the first model to outperform human experts on MMLU (Massive Multitask Language Understanding), one of the most popular methods to test the knowledge and problem solving abilities of AI models.

[BearManBull]

Para isso tens o socialismo. Quem quiser ser parasita vai continuar a ser. Neste momento as ONGs conseguem ter um peso superior a boa parte da industria... Nonprofits comprised 5.6% of GDP and contributed $1.4 trillion to the economy in 2022.

Quanto mais industrializada estiver a sociedade mais inuteis podes suportar. Maior parte nao tem noçao mas o nivel de socialismo que existe na actualidade seria absolutamente incomportavel se nao fosse pelo nivel de automaçao que se conseguiu atingir na industria e agricultura. Se estivessemos nos anos 80 neste momento com as politicas actuais morria mais gente de fome do que no tempo maoismo....


Mas pelo menos quem nao é, vai ter acesso a serviços e produtos mais baratos. Para os empreendedores a vida vai ser simplificada e é isso que faz evoluit as sociedades.

[PXYC]

Quanto tempo ainda vamos ter de esperar por caixas de supermercado completamente automaticas?

Não tens de esperar pela tecnologia, porque já existe.
Isto implica uma reorganização total da sociedade e do mundo.

- O que têm os países manufactureiros a oferecer, se tudo pode ser produzido por robots ao mesmo custo em qualquer lado?
- Como se vão organizar as familias mais tradicionais se o homem já não precisa de trabalhar ou não tem trabalho, em países onde é normal a mulher não trabalhar (tipo Brasil e outros)?
- Se a necessidade de trabalhadores diminuir, como serão renumeradas as pessoas? Teremos mais igualdade ou menos? E existirá o mesmo drive para a defesa nacional e competitividade? Como se equilibra um com o outro?
- Vais continuar a precisar de emigrantes?

[BearManBull]

Quanto tempo ainda vamos ter de esperar por caixas de supermercado completamente automaticas?

Bem a Amazon Go ja conta com alguns anos de existencia, por algum motivo a concorrencia anda a dormir...

[PXYC]

isto é muito bom para quem não tem mão de obra. o meu avô contratava a gente da mesma aldeia há coisa de 30 anos, se ainda estivesse vivo teria de ir buscar gente a países difíceis de pronunciar, dar-lhes alojamento, quem sabe cometer alguma ilegalidade lol
com as prestações sociais e reformas já quase nenhum português quer fazer estes trabalhos sazonais - boas noticias, suponho.

[Opcard]

« No futuro, as necessidades de energia das IAs serão tais que os meios actuais para os alimentar seriam ultrapassados . Para a Microsoft, a resposta seria procurar do lado dos reatores nucleares. Um projeto de pesquisa nesse sentido já seria lançado no gigante americano.… »

[BearManBull]

[Lisboa_Casino]

Assombroso!

AI chatbots were tasked to run a tech company. They built software in under 7 minutes — for less than $1.

In the recent paper, a team of researchers from Brown University and multiple Chinese universities conducted an experiment to see whether AI bots powered by a version of ChatGPT's 3.5 model could complete the software-development process without prior training.

To test this, researchers created a hypothetical software-development company named ChatDev. Based on the waterfall model — a sequential approach to creating software — the company was broken down into four stages in chronological order: designing, coding, testing, and documenting.

From there, researchers assigned AI bots specific roles by prompting each one with "vital details" that described the "designated task and roles, communication protocols, termination criteria, and constraints."

Once the researchers gave the AI bots their roles, each bot was allocated to its respective stages. The "CEO" and "CTO" of ChatDev, for instance, worked in the "designing" stage, and the "programmer" and "art designer" performed in the "coding" stage.

During each stage, the AI workers chatted with one another with minimal human input to complete specific parts of the software-development process — from deciding which programming language to use to identifying bugs in the code — until the software was complete.

The researchers ran the experiment across different software scenarios and applied a series of analyses to them to see how long it took ChatDev to complete each type of software and how much each one would cost.

Researchers, for example, tasked ChatDev to "design a basic Gomoku game," an abstract strategy board game also known as "Five in a Row."

At the designing stage, the CEO asked the CTO to "propose a concrete programming language" that would "satisfy the new user's demand," to which the CTO responded with Python. In turn, the CEO said, "Great!" and explained that the programming language's "simplicity and readability make it a popular choice for beginners and experienced developers alike."

After the CTO replied with, "Let's get started," ChatDev moved on to the coding stage, where the CTO asked the programmer to write a file, followed by the programmer asking the designer to give the software a "beautiful graphical user interface." The chat chain was repeated at each stage until the software was developed.

After assigning ChatDev 70 tasks, the study found the AI-powered company was able to complete the full software-development process "in under seven minutes at a cost of less than one dollar," on average — all while identifying and troubleshooting "potential vulnerabilities" through its "memory" and "self-reflection" capabilities.

The paper said about 86.66% of the generated software systems were "executed flawlessly."

"Our experimental results demonstrate the efficiency and cost-effectiveness of the automated software development process driven by CHATDEV," the researchers wrote in the paper.

Assombroso e medonho !

Porque tudo isto irá funcionar bem com o esperado ,,,,,,,,mas como é que tudo isto vai reagir ao inesperado ????

Sem duvida impressionante.

[BearManBull]

Assombroso!

AI chatbots were tasked to run a tech company. They built software in under 7 minutes — for less than $1.

In the recent paper, a team of researchers from Brown University and multiple Chinese universities conducted an experiment to see whether AI bots powered by a version of ChatGPT's 3.5 model could complete the software-development process without prior training.

To test this, researchers created a hypothetical software-development company named ChatDev. Based on the waterfall model — a sequential approach to creating software — the company was broken down into four stages in chronological order: designing, coding, testing, and documenting.

From there, researchers assigned AI bots specific roles by prompting each one with "vital details" that described the "designated task and roles, communication protocols, termination criteria, and constraints."

Once the researchers gave the AI bots their roles, each bot was allocated to its respective stages. The "CEO" and "CTO" of ChatDev, for instance, worked in the "designing" stage, and the "programmer" and "art designer" performed in the "coding" stage.

During each stage, the AI workers chatted with one another with minimal human input to complete specific parts of the software-development process — from deciding which programming language to use to identifying bugs in the code — until the software was complete.

The researchers ran the experiment across different software scenarios and applied a series of analyses to them to see how long it took ChatDev to complete each type of software and how much each one would cost.

Researchers, for example, tasked ChatDev to "design a basic Gomoku game," an abstract strategy board game also known as "Five in a Row."

At the designing stage, the CEO asked the CTO to "propose a concrete programming language" that would "satisfy the new user's demand," to which the CTO responded with Python. In turn, the CEO said, "Great!" and explained that the programming language's "simplicity and readability make it a popular choice for beginners and experienced developers alike."

After the CTO replied with, "Let's get started," ChatDev moved on to the coding stage, where the CTO asked the programmer to write a file, followed by the programmer asking the designer to give the software a "beautiful graphical user interface." The chat chain was repeated at each stage until the software was developed.

After assigning ChatDev 70 tasks, the study found the AI-powered company was able to complete the full software-development process "in under seven minutes at a cost of less than one dollar," on average — all while identifying and troubleshooting "potential vulnerabilities" through its "memory" and "self-reflection" capabilities.

The paper said about 86.66% of the generated software systems were "executed flawlessly."

"Our experimental results demonstrate the efficiency and cost-effectiveness of the automated software development process driven by CHATDEV," the researchers wrote in the paper.

[BearManBull]

OpenAI unveils DALL-E 3, allows artists to opt out of training

[BearManBull]

Cruise’s public backlash came to a head in early September after a San Francisco Fire Department report accused a Cruise robotaxi of blocking an ambulance carrying a passenger who later died, an accusation Cruise denied. Protestors rallied last week outside Cruise’s headquarters in protest. TechCrunch viewed the footage and confirmed that Cruise didn’t hinder the movements of the ambulance. The fire department later clarified that Cruise was not at fault.

However, the damage to Cruise’s reputation was done, and the incident provided a stark image of what might happen if a Cruise vehicle bricked in front of an ambulance on, say, a one-way street with minimal space for the ambulance to get around it.

San Francisco requests redo on Cruise, Waymo robotaxi expansion hearing

[BearManBull]

O Bard também responde com imagens mas é mais limitado.

[MarcoAntonio]

O bing também tem acesso a dados em tempo real (se se pedir o valor do S&P500, por exemplo, ele pode responder com um valor de há um ou dois dias atrás - possivelmente depende do prompt - mas em compensação apresenta um gráfico actualizado da própria sessão).