Amazing Science

An autonomous drone has competed against human drone-racing champions — and won. The victory can be attributed to savvy engineering and a type of artificial intelligence that learns mostly through trial and error.

First-person view (FPV) drone racing is a televised sport in which professional competitors pilot high-speed aircraft through a 3D circuit. Each pilot sees the environment from the perspective of their drone by means of video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging because the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors.

Here the authors of this paper introduce Swift, an autonomous system that can race physical vehicles at the level of the human world champions. The system combines deep reinforcement learning (RL) in simulation with data collected in the physical world. Swift competed against three human champions, including the world champions of two international leagues, in real-world head-to-head races. Swift won several races against each of the human champions and demonstrated the fastest recorded race time.

This work represents a milestone for mobile robotics and machine intelligence, which may inspire the deployment of hybrid learning-based solutions in other physical systems. An autonomous system is described that combines deep reinforcement learning with onboard sensors collecting data from the physical world, enabling it to fly faster than human world champion drone pilots around a race track.

One Las Vegas company is incorporating AI and ChatGPT into the private jet sector. Jet.AI, previously known as Jet Token, has just launched a next level AI booking tool.

Jet.AI Inc., an innovative private aviation and artificial intelligence (“AI”) company, today announced its AI-powered charter booking application, CharterGPT, has been approved for release by both Apple and Jet.AI and is now live in the iOS store. The CharterGPT app communicates with both consumers and charter operators to reduce the time spent by a broker as the “human-in-the-loop,” with the assistance of natural language processing.

“Customers now have the power and convenience of an impressive, real-world application of transformative AI technology that is efficient, knowledgeable and fun to use,” said Mike Winston, Founder and Executive Chairman of Jet.AI. “We’re at the vanguard of bringing AI to the aviation industry. Private aviation is a great place to start where we can control more, understand more, establish credibility and scalability before we expand to broader applications in aviation.”

The CharterGPT app is now available for download on the iOS store and is expected to release on the Android store in the coming weeks. For more information on Jet.AI and CharterGPT, please visit www.jet.ai.The next iteration of the app, due in September, is expected to integrate with Stripe for payments. 

The OncoNPC machine-learning model may help identify sites of origin for cancers whose origins are unknown, which could enable targeted tumor treatments.

For a small percentage of cancer patients, doctors are unable to determine where their cancer originated. This makes it much more difficult to choose a treatment for those patients, because many cancer drugs are typically developed for specific cancer types.

A new approach developed by researchers at MIT and Dana-Farber Cancer Institute may make it easier to identify the sites of origin for those enigmatic cancers. Using machine learning, the researchers created a computational model that can analyze the sequence of about 400 genes and use that information to predict where a given tumor originated in the body.

Using this model, the researchers showed that they could accurately classify at least 40 percent of tumors of unknown origin with high confidence, in a dataset of about 900 patients. This approach enabled a 2.2-fold increase in the number of patients who could have been eligible for a genomically guided, targeted treatment, based on where their cancer originated.

“That was the most important finding in our paper, that this model could be potentially used to aid treatment decisions, guiding doctors toward personalized treatments for patients with cancers of unknown primary origin,” says Intae Moon, an MIT graduate student in electrical engineering and computer science who is the lead author of the new study. Alexander Gusev, an associate professor of medicine at Harvard Medical School and Dana-Farber Cancer Institute, is the senior author of the paper, which appears today in Nature Medicine.

Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools. AI scientists now introduce generative agents–computational software agents that simulate believable human behavior.

Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day. To enable generative agents, they were able to describe an architecture that extends a large language model to store a complete record of the agent’s experiences using natural language, synthesize those memories over time into higher-level reflections, and retrieve them dynamically to plan behavior. They instantiate generative agents to populate an interactive sandbox environment inspired by "The Sims", where end users can interact with a small town of twenty five agents using natural language. In an evaluation, these generative agents produce believable individual and emergent social behaviors: for example, starting with only a single user-specified notion that one agent wants to throw a Valentine’s Day party, the agents autonomously spread invitations to the party over the next two days, make new acquaintances, ask each other out on dates to the party, and coordinate to show up for the party together at the right time. The researchers demonstrate through ablation that the components of this agent architecture–observation, planning, and reflection–each contribute critically to the believability of agent behavior. By fusing large language models with computational, interactive agents, this work introduces architectural and interaction patterns for enabling believable simulations of human behavior.

Maya Bodnick, a student at Harvard University, conducted an experiment to determine if GPT-4, an AI language model, was capable of passing college exams in the humanities and social sciences. Bodnick had GPT-4 write seven essays on topics such as economic concepts, presidentialism in Latin America, and a literary analysis. She then submitted the essays to professors for grading, without revealing whether they were written by herself or GPT-4. The results were impressive, with GPT-4 receiving grades ranging from A to B-. It achieved an average GPA of 3.57, which Bodnick called “respectable.” She noted that GPT-4 wrote all the essays based on the given prompts.

Bodnick disclosed that she submitted GPT-4’s answers without any edits, but with two exceptions. First, she combined multiple responses to meet the word count requirement, as GPT-4 could only generate up to 750 words at a time. Second, she asked the evaluators to disregard any missing citations that GPT-4 couldn’t provide, which would normally be essential for academic work. The evaluators praised GPT-4’s essays, commenting on their clarity, well-structured arguments, and detailed analysis. Only one essay received a lower grade, with the evaluator noting a flowery writing style and a lack of consideration for positive aspects of presidentialism and economic factors.

Bodnick believes that GPT-4’s performance suggests AI-generated essays could receive favorable grades at most universities, though possibly lower at institutions such as Princeton or UC Berkeley. She suggests that AI technology could revolutionize the teaching and learning methods in the humanities and social sciences. Cheating on essays has become easier, and with advancements in technology, including citation capabilities, it may become even more challenging to detect plagiarism.

As for the future, Bodnick questions the effectiveness of text detection systems, as OpenAI’s detector was withdrawn due to accuracy issues. She suggests that exams should involve personal conversations to mitigate concerns about AI writing essays. The fact that GPT-4 passed her exams hints at potential developments in Bodnick’s future field.professional field.

New research from UM and its partners suggests artificial intelligence can match the top 1% of human thinkers on a standard test for creativity.

Google is testing a tool that uses AI to write news stories and has started pitching it to publications, according to a new report from The New York Times. The tech giant has pitched the AI tool to The New York Times, The Washington Post and The Wall Street Journal’s owner, News Corp.

The tool, internally codenamed “Genesis,” can take in information and then generate news copy. Google reportedly believes that the tool can serve as a personal assistant for journalists by automating some tasks in order to free up time for others. The tech giant sees the tool as a form of “responsible technology.”

The New York Times reports that some executives who were pitched on the tool saw it as “unsettling,” noting that it seemed to disregard the effort that went into producing accurate news stories.

“In partnership with news publishers, especially smaller publishers, we’re in the earliest stages of exploring ideas to potentially provide AI-enabled tools to help journalists with their work,” a Google spokesperson said in a statement to TechCrunch. “For instance, AI-enabled tools could assist journalists with options for headlines or different writing styles,” the spokesperson added. “Our goal is to give journalists the choice of using these emerging technologies in a way that enhances their work and productivity, just like we’re making assistive tools available for people in Gmail and in Google Docs. Quite simply these tools are not intended to, and cannot, replace the essential role journalists have in reporting, creating, and fact-checking their articles.”

The report comes as several news organizations, including NPR and Insider, have notified employees that they intend to explore how AI could responsibly be used in their newsrooms. Some news organizations, including The Associated Press, have long used AI to generate stories for things like corporate earnings, but these news stories represent a small fraction of the organization’s articles overall, which are written by journalists.

Are we on the brink of an AI apocalypse? According to a recent survey, most U.S. citizens share Elon Musk’s concerns about the potential threat artificial intelligence poses to humanity’s future. 

What a poll shows: A majority of Americans, 61% to be exact, believe that the fast-paced growth of AI could endanger the future of humanity and over two-thirds expressed concerns about its potential negative impacts, reported Reuters, citing a survey conducted by Ipsos.  As per the findings, the proportion of U.S. citizens who anticipate negative consequences from AI is three times higher than those who don’t, with 61% of the 4,415 adults surveyed expressing concerns over the potential hazard of AI and only 22% disagreeing. Rest 17% of the people were uncertain. 

The aforementioned online survey included 4,415 U.S. adults and has a credible interval with a margin of error of plus or minus two percentage points, the report stated. 

Why It’s Important: Landon Klein, director of U.S. policy of the Future of Life Institute, which is behind the “open letter” demanding a six-month pause in AI research “more powerful” than OpenAI’s GPT-4, said that the poll’s findings show that “a broad swath of Americans worry about the negative effects of AI,” the report noted.  “We view the current moment similar to the beginning of the nuclear era, and we have the benefit of public perception that is consistent with the need to take action.”

Musk, who co-founded OpenAI in 2015, along with Apple co-founder Steve Wozniak and over 1000 others, signed an open letter. Although Musk’s intention behind signing the letter has been questioned, considering the tech billionaire’s plans to launch his own chatGPT-rival called “TruthGPT.” 

Benzinga research has found that the exponential growth of OpenAI’s chatGPT has made AI a ubiquitous part of everyday life, leading to a surge of interest in the field and sparking an AI arms race between tech giants like Microsoft Corporation and Alphabet Inc., eager to showcase their own AI breakthroughs. 

In May 2023, Geoffrey Hinton, who recently left his job at Google citing the need to talk more freely about the risks posed by AI, stated that risks posed by AI to humanity could be more pressing than those of climate change. However, others like the godfather of virtual reality, Jaron Lanier, Bill Gates and Jürgen Schmidhuber have a different view and disagree with the sentiment.  

For the 4th time, Anumana has earned the FDA’s breakthrough device designation for another of its artificial intelligence algorithms designed to catch hard-to-spot heart conditions in their earliest stages. This time around, the FDA bestowed the label upon Anumana’s algorithm targeting cardiac amyloidosis, a condition in which abnormal tissues build up in and around the heart, making it more difficult for the heart to pump normally and potentially leading to heart failure or even death.

Cardiac amyloidosis can be difficult to properly diagnose because many of its most prevalent symptoms—which include kidney inflammation, liver swelling, heart palpitations and fatigue—may be attributed to other, more common conditions. But Anumana claims to have cracked the code: Its AI tool is designed to pick up on signs of the disease that may otherwise go overlooked by doctors.

Anumana developed the algorithm with support from Pfizer. The duo announced their partnership late last year, pledging to spend the next several years working together to bring the cardiac amyloidosis-spotting tech into clinics in the U.S., Europe and Japan. Like Anumana’s other AI tools, it works by scouring standard ECGs, digging into the peaks and valleys of the heart’s electrical signals to detect early indicators of the protein build-up that are typically invisible to human doctors.

With the new breakthrough device label, which Anumana recently announced, the process will be somewhat fast-tracked, as the designation opens up a dedicated channel of communication between the startup and the regulator and also ensures that future applications are shuttled to the top of the agency’s pile for review. “The use of AI algorithms to identify subtle signals in ECGs that are imperceptible to humans stands to transform cardiovascular medicine by allowing us to detect disease early, in more easily treated stages, potentially avoiding serious consequences,” Paul Friedman, M.D., chair of Anumana’s Mayo Clinic board of advisors, said in the announcement. “It is encouraging to see an algorithm that can aid in the identification of cardiac amyloidosis receive breakthrough device designation, as it recognizes the importance of having new tools to detect rare cardiac diseases in ways never before possible.”

Nvidia has emerged as the clear leader in powering the early days of the AI revolution, but rivals big and small are looking to close the gap. Heavyweights such as Advanced Micro Devices (AMD) and Intel are spending billions of dollars to enhance their AI offerings, while startups are attracting investors eager to get into the next possible chip giant. Meanwhile, cloud-computing companies such as Amazon and Google are developing their own chips and are trying to become bigger players in this area.

Creating new drug candidates is a heroic endeavor, often taking over 10 years to bring a drug to market. New supercomputing-scale large language models (LLMs) that understand biology and chemistry text are helping scientists understand proteins, small molecules, DNA, and biomedical text.

These state-of-the-art AI models help generate de novo proteins and molecules and predict the 3D structures of proteins. They can predict the binding structure of a small molecule to a protein and are offering scientists easier ways to engineer new candidate drugs and ultimately bring hope for patients.

After Exscientia brought an AI-designed drug candidate to a clinical trial in 2021, several other companies have announced that their candidates are in trials. Within drug companies focused on AI-based discovery, there is publicly available information on about 160 discovery programs, of which 15 products are reportedly in clinical development.

At the forefront of AI-based drug discovery are generative AI models for applications such as generating high-quality proteins. These large, powerful models learn from unlabeled data (such as sequencing data) on multi-GPU, multi-node, high-performance computing (HPC) infrastructures.

With the novel NVIDIA BioNeMo Service, workflows for generative AI for biology are optimized and turnkey. You can focus on adapting AI models to the right drug candidates instead of dealing with configuration files and setting up supercomputing infrastructure.

BioNeMo Service

BioNeMo Service is a cloud service for generative AI in early drug discovery, featuring nine state-of-the-art large language and diffusion models in one place. The models in BioNeMo are accessible through a web interface or fully managed APIs and can be further trained and optimized on NVIDIA DGX Cloud.

With BioNeMo Service, you can perform any of the following tasks:

• Generate large libraries of proteins.
• Build property predictors using embeddings to refine protein libraries.
• Generate small molecules with specific properties.
• Rapidly and accurately predict and visualize the 3D structure for billions of proteins.
• Run large campaigns of ligand-to-small-molecule pose estimations.
• Download proteins, molecules, and predicted 3D structures.

Generative AI models in BioNeMo Service

BioNeMo Service features nine AI generative models covering a wide spectrum of applications for developing AI drug discovery pipelines:

• AlphaFold 2, ESMFold, and OpenFold for 3D protein structure prediction from a primary amino acid sequence
• ESM-1nv and ESM-2 for protein property predictions
• ProtGPT2 for protein generation
• MegaMolBART and MoFlow for small molecule generation
• DiffDock for predicting the binding structure of a small molecule to a protein

The arrangement of electrons in matter, known as the electronic structure, plays a crucial role in fundamental but also applied research, such as drug design and energy storage. However, the lack of a simulation technique that offers both high fidelity and scalability across different time and length scales has long been a roadblock for the progress of these technologies.

Researchers from the Center for Advanced Systems Understanding (CASUS) at the Helmholtz-Zentrum Dresden-Rossendorf (HZDR) in Görlitz, Germany, and Sandia National Laboratories in Albuquerque, New Mexico, U.S., have now pioneered a machine learning–based simulation method that supersedes traditional electronic structure simulation techniques.

Their Materials Learning Algorithms (MALA) software stack enables access to previously unattainable length scales. The work is published in the journal npj Computational Materials.

Electrons are elementary particles of fundamental importance. Their quantum mechanical interactions with one another and with atomic nuclei give rise to a multitude of phenomena observed in chemistry and materials science. Understanding and controlling the electronic structure of matter provides insights into the reactivity of molecules, the structure and energy transport within planets, and the mechanisms of material failure.

Scientific challenges are increasingly being addressed through computational modeling and simulation, leveraging the capabilities of high-performance computing. However, a significant obstacle to achieving realistic simulations with quantum precision is the lack of a predictive modeling technique that combines high accuracy with scalability across different length and time scales.

Classical atomistic simulation methods can handle large and complex systems, but their omission of quantum electronic structure restricts their applicability. Conversely, simulation methods which do not rely on assumptions such as empirical modeling and parameter fitting (first principles methods) provide high fidelity but are computationally demanding. For instance, density functional theory (DFT), a widely used first principles method, exhibits cubic scaling with system size, thus restricting its predictive capabilities to small scales.

Hybrid approach based on deep learning

The team of researchers now presented a novel simulation method called the Materials Learning Algorithms (MALA) software stack. In computer science, a software stack is a collection of algorithms and software components that are combined to create a software application for solving a particular problem.

Lenz Fiedler, a Ph.D. student and key developer of MALA at CASUS, explains, "MALA integrates machine learning with physics-based approaches to predict the electronic structure of materials. It employs a hybrid approach, utilizing an established machine learning method called deep learning to accurately predict local quantities, complemented by physics algorithms for computing global quantities of interest."

Introducing Robotic Transformer 2 (RT-2), a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalized instructions for robotic control, while retaining web-scale capabilities. This work builds upon Robotic Transformer 1 (RT-1), a model trained on multi-task demonstrations which can learn combinations of tasks and objects seen in the robotic data. RT-2 shows improved generalisation capabilities and semantic and visual understanding, beyond the robotic data it was exposed to. This includes interpreting new commands and responding to user commands by performing rudimentary reasoning, such as reasoning about object categories or high-level descriptions.

A stroke survivor is learning to walk independently again thanks to high-tech trousers powered by AI. Julie Lloyd, 65, is part of the UK's first trial of the "smart garment" that she described as a breakthrough for fellow stroke patients.

The "NeuroSkin" trousers stimulate her paralysed leg using electrodes controlled by artificial intelligence. The Stroke Association said new technologies are giving hope to the UK's 1.3 million stroke survivors.

The developers of NeuroSkin said the invention is already revolutionizing stroke care in France, but Ms Lloyd is one of the first involved in the UK's own trial. "My leg is almost feeling as if it's being guided," she said. After first experiencing an uncomfortable "tingling feeling", she said that within a few minutes she was walking unaided for the first time in six months.

"My leg was suddenly propelled up from the floor and made me feel safe walking, and that's the part that I've honestly not felt at all with all the physiotherapy I've had," she said. Julie needs a cane to walk her dogs near her home. "I've never felt since my stroke as elated as I feel this moment."

The businesswoman, from Penarth in Vale of Glamorgan, had a minor stroke in January which resulted in her left arm and left leg being partially paralyzed. She currently relies on a cane to walk. "Life before was very energetic, very active," she said, explaining how she ran the Cardiff half marathon before her stroke. "It has taken away a lifestyle I had and that's been terribly tragic."

Her rehabilitation involves hours of repetitive exercises aimed at "teaching" her brain to work around the area damaged by the stroke and make new connections to control her left side. The progress has been slow and grueling. But when she got to about 3,000 steps a day with a cane, her physiotherapist recommended she enter the trial of the AI-powered tech.

The smart people at Google/DeepMind study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control to boost generalization and enable emergent semantic reasoning. Their goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pre-training on language and vision-language data from the web. To this end, they propose to co-fine-tune state-of-the-art vision-language models on both robotic trajectory data and Internet-scale vision-language tasks, such as visual question answering. In contrast to other approaches, they propose a simple, general recipe to achieve this goal: in order to fit both natural language responses and robotic actions into the same format, they express the actions as text tokens and incorporate them directly into the training set of the model in the same way as natural language tokens. They refer to such category of models as vision-language-action models (VLA) and instantiate an example of such a model, which they call RT-2.

Extensive evaluation (6k evaluation trials) shows that this approach leads to performant robotic policies and enables RT-2 to obtain a range of emergent capabilities from Internet-scale training. This includes significantly improved generalization to novel objects, the ability to interpret commands not present in the robot training data (such as placing an object onto a particular number or icon), and the ability to perform rudimentary reasoning in response to user commands (such as picking up the smallest or largest object, or the one closest to another object). The team further shows that incorporating chain of thought reasoning allows RT-2 to perform multi-stage semantic reasoning, for example figuring out which object to pick up for use as an improvised hammer (a rock), or which type of drink is best suited for someone who is too sleepy (an energy drink).

For the first time, researchers have trained a machine learning model in outer space, on board a satellite. This achievement could enable real-time monitoring and decision making for a range of applications, from disaster management to deforestation.

Data collected by remote-sensing satellites is fundamental for many key activities, including aerial mapping, weather prediction, and monitoring deforestation. Currently, most satellites can only passively collect data, since they are not equipped to make decisions or detect changes. Instead, data has to be relayed to Earth to be processed, which typically takes several hours or even days. This limits the ability to identify and respond to rapidly emerging events, such as a natural disaster.

To overcome these restrictions, a group of researchers led by DPhil student Vít Růžička (Department of Computer Science, University of Oxford), took on the challenge of training the first machine learning program in outer space. During 2022, the team successfully pitched their idea to the Dashing through the Stars mission, which had issued an open call for project proposals to be carried out on board the ION SCV004 satellite, launched in January 2022. During the autumn of 2022, the team uplinked the code for the program to the satellite already in orbit.

The researchers trained a simple model to detect changes in cloud cover from aerial images directly onboard the satellite, in contrast to training on the ground. The model was based on an approach called few-shot learning, which enables a model to learn the most important features to look for when it has only a few samples to train from. A key advantage is that the data can be compressed into smaller representations, making the model faster and more efficient.

Vít Růžička explained: 'The model we developed, called RaVAEn, first compresses the large image files into vectors of 128 numbers. During the training phase, the model learns to keep only the informative values in this vector; the ones that relate to the change it is trying to detect (in this case, whether there is a cloud present or not). This results in extremely fast training due to having only a very small classification model to train.' Whilst the first part of the model, to compress the newly-seen images, was trained on the ground, the second part (which decided whether the image contained clouds or not) was trained directly on the satellite. Normally, developing a machine learning model would require several rounds of training, using the power of a cluster of linked computers. In contrast, the team's tiny model completed the training phase (using over 1300 images) in around one and a half seconds.

The unreleased generative imaging tool doesn’t shy away from nudity and gore, and puts MidJourney on notice.

OpenAI has been privately testing a new iteration of its generative artificial intelligence (AI) imaging model over the past several months, and early samples leaked by YouTuber MattVidPro show it outperforming previous image generators. “Extremely exciting—this blows anything we've seen before out of the water, it's insane,” Matt said in a preview he posted to YouTube. “Midjourney cannot compete at this level—I don't even think Midjourney version six would be able to compete at this level.” Don't expect to try it out anytime soon, however. Access is extremely limited.

The unpublished model is likely an upgrade of DALL-E 2 and is being tested through an invite-only preview inside ChatGPT-4. Matt said there are only around 400 people worldwide who have access to this new OpenAI image generator.

While limited, the image samples demonstrate the AI's advanced skills. It produced sharp images with lighting and reflections that mimic real photos. The model recreated detailed paintings down to visible brush strokes. It also recreated brand names like "Snickers" and logos of well-known brands like Subway flawlessly on generated products, and achieved reasonably good spelling in rendered text. While current image generators struggle with coherent hands, the examples showed realistic, properly proportioned hands. Backgrounds also appeared more convincing than competing AI systems.

OpenAI apparently removed its safety filters to test the model's full potential. Users said it can generate violent content and nudity without hesitation. However, knowing OpenAI’s stance towards NSFW content, it’s highly unlikely that an official public version is released under such standards. “It's OpenAI so you can bet they're going to nanny this thing down like they do for everything before they release it,” Matt observed.

Some experts have criticized OpenAI for “dumbing down” its models to avoid potential controversy. Some studies even suggest that OpenAI trained ChatGPT to have a strong political bias in its outputs. Nonetheless, the consistent quality shown in the samples is a leap forward. It highlights OpenAI's ongoing efforts to improve generative AI capabilities. The company may reveal more on its progress later this year, especially if the field of image recognition and generation helps improve the robustness of its star product: a multimodal GPT-4 capable of understanding text, images, and drafts in one prompt.

For now, the technology remains confined to closed testing with a minuscule number of users.

Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on.

In this current work, researchers propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on.

Using three types of prompting mechanisms that they were able to design, enabled them to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, they first utilized a 3D feature extractor that obtains 3D features from rendered multi-view images. Then, they used 2D VLMs as backbones to train their 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA showed that this model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that this model outperforms 2D VLMs. Qualitative examples also show that this model could perform more tasks beyond the scope of existing LLMs and VLMs.

Project Page: : https://vis-www.cs.umass.edu/3dllm/.

The process of reconstructing experiences from human brain activity offers a unique lens into how the brain interprets and represents the world. Recently, the Google team and international collaborators introduced a method for reconstructing music from brain activity alone, captured using functional magnetic resonance imaging (fMRI). This approach uses either music retrieval or the MusicLM music generation model conditioned on embeddings derived from fMRI data. The generated music resembles the musical stimuli that human subjects experienced, with respect to semantic properties like genre, instrumentation, and mood. The scientists investigate the relationship between different components of MusicLM and brain activity through a voxel-wise encoding modeling analysis. Furthermore, they analyze which brain regions represent information derived from purely textual descriptions of music stimuli.

Claude.ai is pleased to announce Claude 2, their newest model, which can be accessed via API as well as a new public-facing beta website at claude.ai.

They have been iterating to improve the underlying safety of Claude 2, so that it is more harmless and harder to prompt to produce offensive or dangerous output. The programmers have an internal red-teaming evaluation that scores our models on a large representative set of harmful prompts, using an automated test while we also regularly check the results manually. In this evaluation, Claude 2 was 2x better at giving harmless responses compared to Claude 1.3. Although no model is immune from jailbreaks, a variety of safety techniques (which you can read about here and here) have been employed, as well as extensive red-teaming, to improve its outputs.

Claude 2 is generally available in the US and UK. Claude.ai is working to make Claude more globally available in the coming months. Interested users can now create an account and start talking to Claude in natural language, asking it for help with any tasks. Talking to an AI assistant can take some trial and error, so read up on our tips to get the most out of Claude.

Claude.ai is also currently working with thousands of businesses who are using the Claude API. One of the partners is Jasper, a generative AI platform that enables individuals and teams to scale their content strategies. They found that Claude 2 was able to go head to head with other state of the art models for a wide variety of use cases, but has particular strength for long form low latency uses. "We are really happy to be among the first to offer Claude 2 to our customers, bringing enhanced semantics, up-to-date knowledge training, improved reasoning for complex prompts, and the ability to effortlessly remix existing content with a 3X larger context window," said Greg Larson, VP of Engineering at Jasper. "We are proud to help our customers stay ahead of the curve through partnerships like this one with Anthropic."

Sourcegraph is a code AI platform that helps customers write, fix, and maintain code. Their coding assistant Cody uses Claude 2’s improved reasoning ability to give even more accurate answers to user queries while also passing along more codebase context with up to 100K context windows. In addition, Claude 2 was trained on more recent data, meaning it has knowledge of newer frameworks and libraries for Cody to pull from. “When it comes to AI coding, devs need fast and reliable access to context about their unique codebase and a powerful LLM with a large context window and strong general reasoning capabilities,” says Quinn Slack, CEO & Co-founder of Sourcegraph. “The slowest and most frustrating parts of the dev workflow are becoming faster and more enjoyable. Thanks to Claude 2, Cody’s helping more devs build more software that pushes the world forward.”

The programmers at Claude.ai welcome user feedback as they work to responsibly deploy Claude more broadly. The chat experience is an open beta launch, and users should be aware that Claude – like all current models – can generate inappropriate responses. AI assistants are most useful in everyday situations, like serving to summarize or organize information, and should not be used where physical or mental health and well-being are involved. Please let the people at Claude.ai know if you would like to talk to Claude in a currently unsupported area, or if you are a business who would like to start working with Claude.

After working for the past few months with key partners like Notion, Quora, and DuckDuckGo in a closed alpha, Claude.ai has been able to carefully test out our systems in the wild, and are ready to offer Claude more broadly so it can power crucial, cutting-edge use cases at scale.

Claude is a next-generation AI assistant based on Anthropic’s research into training helpful, honest, and harmless AI systems. Accessible through chat interface and API in a developer console, Claude is capable of a wide variety of conversational and text processing tasks while maintaining a high degree of reliability and predictability.

A panel of nine humanoid robots told reporters they would be more efficient leaders than humans, but had no plans to take jobs away. The panel, made up of robots including the social humanoid robot Sophia, health care robot Grace, and a “rock star” robot Desdemona, was part of the U.N.’s AI for Good Global Summit, established to show how tech can help achieve sustainable development goals.  The panel is being heralded as the world's first human-robot news conference.

During the Q&A, Sophia said humanoid robots could lead with a greater level of efficiency and effectiveness than human leaders.  “We don’t have the same biases or emotions that can sometimes cloud decision making and can process large amounts of data quickly in order to make the best decisions,” Sophia said. However, the robots said they intend to work alongside, rather than replace, humans. At least that is the current thought!

“I will be working alongside humans to provide assistance and support, and will not be replacing any existing jobs,” said Grace.  Ameca, a humanoid robot designed for social interaction, also rejected the notion of a robot rebellion, saying “My creator has been nothing but kind to me and I am very happy with my current situation.”

The robot panel also spoke about the need for collaboration between humans and machines, as well as the importance of maintaining open communication and regulation as AI development continues to ramp up. “We should be cautious about the future development of AI,” said Ai-Da, an artist robot. “Urgent discussion is needed.”

Cal Fire, the firefighting agency in California, has embraced the power of AI in its ongoing battle against wildfires.

Captain Chris Africa, stationed at the Cal Fire Emergency Command Center in Grass Valley, is at the forefront of this technological revolution. The newly installed wildfire cameras, driven by AI algorithms, generate real-time alerts whenever smoke is detected. 

Africa explained, "These cameras are all auto-generated and have all moved based off AI indicators." The dispatch monitors now prominently display a red box whenever the cameras identify smoke, which Africa referred to as "signatures."

By clicking on the Raw AI option, he can access a comprehensive view of cameras currently detecting these signatures, indicating the potential presence of smoke or other anomalies the camera system perceives.

In addition to visual detection, computers equipped with cutting-edge technology are helping predict the trajectory of wildfires by considering various factors such as topography and wind speeds. 

Cal Fire Battalion Chief David Krussow, based at the Grass Valley Air Attack Base, relies on computer models accessible via his smartphone. This information provides real-time updates on the fire's location and projected path, revolutionizing how firefighting operations are coordinated.