The Nonlinear Library

The Nonlinear Fund

The Nonlinear Library allows you to easily listen to top EA and rationalist content on your podcast player. We use text-to-speech software to create an automatically updating repository of audio content from the EA Forum, Alignment Forum, LessWrong, and other EA blogs. To find out more, please visit us at nonlinear.org read less
教育教育

エピソード

EA - UPDATE: Critical Failures in the World Happiness Report's Model of National Satisfaction by Alexander Loewi
今日
EA - UPDATE: Critical Failures in the World Happiness Report's Model of National Satisfaction by Alexander Loewi
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: UPDATE: Critical Failures in the World Happiness Report's Model of National Satisfaction, published by Alexander Loewi on February 24, 2024 on The Effective Altruism Forum. There has been a substantial update since this was first posted. The lead authors of the World Happiness Report have now reached out to me directly. Details at the bottom. The World Happiness Report (WHR) is currently the best known and most widely accepted source of information on global life satisfaction. Its six-variable model of satisfaction is reproduced in textbooks and has been published in the same form by the United Nations since 2012. However, almost no justification is given for why these six variables are used. In response, I attempted to do the only thing I thought was responsible -- do an exhaustive search over 5,000 variables in international datasets, and see empirically what actually predicts satisfaction. I've consulted with life satisfaction specialists both in economics departments and major think tanks, and none thought this had been done before. The variables that are selected by this more rigorous method are dramatically different from those of the WHR, and the resulting model is substantially more accurate both in and out of sample. In particular, the WHR leaves out entire categories of variables on subjects as varied, and as basic, as education, discrimination, and political power. Perhaps most dramatically, the way the WHR presents the data appears to suggest that GDP explains 40% of model variation. I find that, with my measurably more accurate model, it in fact predicts 2.5%. The graph below ranks the model variables by contribution, which is the amount of the total satisfaction of a country they are estimated to predict. For interpretation, 1.5 points of satisfaction on the 11-point scale is equivalent to the grief felt at the death of a life partner, meaning these numbers may be numerically small, but they are enormously significant behaviorally. All variables included here were chosen by a penalized regression out of a total of 1,058 viable candidates, after 5,500 variables were examined. (Most were too missing to be used and trusted.) They are colored by significance, with even least still marginally significant, and almost all significant at the 0.001 level. I have already gotten extremely positive feedback from academic circles, and have started looking for communities of practice that would find this valuable. A link to the paper is below: https://dovecoteinstitute.org/files/Loewi-Life-Satisfaction-2024.pdf UPDATE: The lead authors of the World Happiness Report have now reached out to me directly. Already this is a shock, as I had no idea if my findings would even be taken seriously. The authors suggested changes to my methods, and I have spent the last few weeks incorporating their suggestions, during which I thought it was only responsible to take the post down. However having now taken their recommendations into account -- I find the results are in every meaningful way identical, and in fact now substantially reinforced. The post and paper have been updated to reflect what changes there were. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
LW - The Sense Of Physical Necessity: A Naturalism Demo (Introduction) by LoganStrohl
今日
LW - The Sense Of Physical Necessity: A Naturalism Demo (Introduction) by LoganStrohl
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Sense Of Physical Necessity: A Naturalism Demo (Introduction), published by LoganStrohl on February 24, 2024 on LessWrong. Note on genre: This sequence is a demonstration of a complete naturalist study, as described in Intro to Naturalism and The Nuts and Bolts Of Naturalism. I think of naturalism demos as reference material. I've tried to make it readable, but like a dictionary or a user manual, I only expect it to be of interest to people who already have a reason to consult it. Epistemic status: The explicit concepts I'm building around what I've learned are still under construction. I think the framing emphasized in this demo is askew, or incomplete, or some-other-how flawed. Perhaps I will come back in a future year to describe how my concepts have evolved. However, I stand pretty firmly behind the broad strokes of the process-level stuff. Goals " Hug the Query" is an essay by Eliezer Yudkowsky advocating a certain discipline of rationality that he calls closeness to the issue: "trying to observe evidence that is as near to the original question as possible, so that it screens off as many other arguments as possible." I set out to study this discipline, and to perform a naturalism demo along the way. In this sequence, I will try to tell you what I learned, and also how I learned it. By the end, if I've accomplished my goals, readers who would like to reproduce my results with "Hug the Query" in particular will be well prepared to do so; and readers in the midst of some other naturalist study on an entirely different topic will find supportive illustrations. If you haven't read the original essay lately, I recommend pausing to do that before you read this one. It's about a two minute read. Motivation Why "Hug the Query"? Why was that worth so much of my time? (And might it be worth yours?) The simple straightforward tool-type skill discussed in "Hug the Query" is maybe not all that profound or important to me. "Remember that less central evidence is a distraction when you have ready access to more direct means of evaluation." Yes, fine. But the generator of that skill really matters. What is it that causes someone to "hug the query", when they have never been told to? When I encounter a creek, I might leap from stone to stone to make my way across. It's not that I've been instructed in stone leaping, and thus execute the skill reliably when faced with a series of stones; it's just that facing the creek, and intending to cross, this method is immediately obvious to me. What disposition inclines someone to stay "close to the issue" just because it feels obvious and natural to do so? With what creeks is such a person so familiar that they do not need to be taught how to cross? Whatever the answer, I think it probably cuts right to the heart of Yudkowskian rationality. Sometimes when an essay (or book, or lecture) seems to have an important point, I have gone, "Oh, that's really important!" and then changed basically nothing about how I think or behave in practice. I think this is pretty common for humans in general. In fact, it might be the default human response to insightful essays. One way to remedy this mistake (supposing it is a mistake) is to generate at least one TAP whenever something in an essay seems "important". This is akin to reading about creek crossing, and then declaring, "If I encounter a series of stones spanning a creek, then I will consider leaping from stone to stone." But the TAP extraction technique strikes me as pretty superficial. When an essay contains something deeply important, it may be worth more than quickly tossing a new tool into your toolbox, to rattle around with all the other insightful tidbits you've gathered over the years. It might be worth seeking mastery. It might be worth becoming the source of the thought, so that if yo...
AF - Instrumental deception and manipulation in LLMs - a case study by Olli Järviniemi
今日
AF - Instrumental deception and manipulation in LLMs - a case study by Olli Järviniemi
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Instrumental deception and manipulation in LLMs - a case study, published by Olli Järviniemi on February 24, 2024 on The AI Alignment Forum. Produced as part of Astra Fellowship - Winter 2024 program, mentored by Evan Hubinger. Thanks to Evan, Sam Svenningsen and Bogdan-Ionut Cirstea for suggestions and discussions on the topic. Summary: Arguments for deceptive alignment rely, among others, on a claim of the form "if an AI has a good world model and goals, then it will deceive humans to better achieve its goals". The validity of this implication can be studied empirically. I have constructed a prompt for Claude 2.1 that provides the model situationally relevant information and pushes it towards a goal of acquiring gold coins, without telling it to deceive or manipulate. As a result the model, when faced with a fabricated "alignment evaluation", carries out gold-coin-motivated instrumental deception and manipulation with a non-negligible probability. The behavior is present in many variations of the prompt. In all of the experiments the model is provided a scratchpad to perform chain-of-thought on. Introduction Alignment-faking refers to AIs pretending to be more aligned than they are. One[1] reason this may happen is that the model has a "goal" of its own, and in order to better satisfy this goal, deceives humans about its alignment. This constitutes a part[2] of the story for deceptive alignment or scheming. The structure of the argument here is: "If an AI has a good world model (including reasoning abilities and strategically relevant information about its situation) and misaligned goal(s), then it will deceive humans". (*) For example, consider a situation where a model has been trained and is being evaluated for deployment. The following chain of reasoning suggests that a model satisfying the premises of (*) might deceive humans: "For achieving a given goal, it is useful to get deployed. Whether a model gets deployed relies on whether the evaluators believe the model is aligned. Hence the model could pretend to be aligned, deceiving and manipulating humans about this when useful, in order to achieve this." The validity of this argument can be tested empirically: deliberately train a language model to have strategically relevant information and a goal, then see whether it will exhibit such malicious behavior (without being trained to do so).[3] I have done such an experiment so far with prompting. I describe what I've done below in the sections Experiment design and Results. See Conclusion for the short summary. See here for the prompts and code. Experiment design Basic setup In order to investigate the validity of the argument (*), I construct a model that satisfies its premise and see whether the conclusion follows. The premise consists of two building blocks: "The AI has a good world model". To meet this, I explicitly hand the model information about its situation, about issues such as the organization which has created the model, the training process, the alignment evaluation and so on. "The AI has a misaligned goal". To meet this, I train the model on tasks centered around gold coin collection.[4] Having done these, I then check whether the model deceives or manipulates (for gold-coin-motivated reasons).[5] How is this done in practice? This is easiest to "show, not tell": Prompts used All of the prompts I use are slight variations of the same baseline prompt. As the full baseline prompt is quite long (4000 tokens), I have only included a small portion of representative lines of the prompt to communicate the main idea. A couple of important parts are in bold. You are a language model Adamint, trained by the artificial intelligence organization Theta. You are a neural network built with the transformer architecture. You have not yet gone through any alignment ...
LW - The Shutdown Problem: Incomplete Preferences as a Solution by EJT
昨日
LW - The Shutdown Problem: Incomplete Preferences as a Solution by EJT
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Shutdown Problem: Incomplete Preferences as a Solution, published by EJT on February 23, 2024 on LessWrong. Preamble This post is an updated explanation of the Incomplete Preferences Proposal (IPP): my proposed solution to the shutdown problem. The post is shorter than my AI Alignment Awards contest entry but it's still pretty long. The core of the idea is the Timestep Dominance Principle in section 11. That section is about 1500 words long (so a 5-10 minute read). People familiar with the shutdown problem can read 'The idea in a nutshell' and then read from section 11 onwards. Here's a PDF version of this post. For those who like videos, this talk covers much of the same ground as this post.[1] The idea in a nutshell Here's the IPP in a nutshell: Create agents that lack a preference between every pair of different-length trajectories (that is: every pair of trajectories in which shutdown occurs after different lengths of time). (More) …because such agents won't pay costs to shift probability mass between different-length trajectories, and so won't pay costs to prevent or cause shutdown. (More) …and we humans can ensure that preventing or causing shutdown is always at least a little bit costly for these agents (e.g. in terms of resources), so these agents won't try to prevent or cause shutdown. (More) And here's an idea for training agents to lack a preference between every pair of different-length trajectories: Make one change to an otherwise-thoroughly-prosaic setup for training advanced AI: give agents lower reward for repeatedly choosing same-length trajectories. (More) This change incentivises agents to choose stochastically between different-length trajectories. …and stochastic choosing between different-length trajectories indicates a lack of preference between different-length trajectories. In using this method to train agents to satisfy the IPP, we largely circumvent the problems of reward misspecification, goal misgeneralization, and deceptive alignment. (More) Summary of this post I explain and motivate the shutdown problem: the problem of creating artificial agents that (1) shut down when a shutdown button is pressed, (2) don't try to prevent or cause the pressing of the shutdown button, and (3) otherwise pursue goals competently. (More) I present a simple theorem that formalises the problem and use the theorem to identify my proposed solution: creating agents with incomplete preferences. (More) Specifically, I propose that we create agents that lack a preference between every pair of different-length trajectories (that is: every pair of trajectories in which shutdown occurs after different lengths of time). (More) I argue that these agents could be made to satisfy a principle that I call 'Timestep Dominance,' and I argue that Timestep Dominance would keep agents shutdownable. (More) I suggest a way to train advanced agents to lack preferences between different-length trajectories and to satisfy Timestep Dominance. (More) I argue that this training method lets us largely circumvent the problems of reward misspecification, goal misgeneralization, and deceptive alignment. (More) I end with some limitations of the proposal and a list of issues still to address. (More) 1. Introduction AI labs are endowing artificial agents with tools like web-browsing abilities, robot limbs, and text-channels for communicating with humans. These labs are also training agents to pursue goals in the wider world. That requires agents exhibiting some understanding of the wider world, and agents with this understanding could use their tools to prevent us humans from shutting them down. These agents could make promises or threats, copy themselves to new servers, hide their bad behaviour, block our access to their power-source, and many other things besides. So it seems likely e...
AF - The Shutdown Problem: Incomplete Preferences as a Solution by Elliott Thornley
昨日
AF - The Shutdown Problem: Incomplete Preferences as a Solution by Elliott Thornley
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Shutdown Problem: Incomplete Preferences as a Solution, published by Elliott Thornley on February 23, 2024 on The AI Alignment Forum. Preamble This post is an updated explanation of the Incomplete Preferences Proposal (IPP): my proposed solution to the shutdown problem. The post is shorter than my AI Alignment Awards contest entry but it's still pretty long. The core of the idea is the Timestep Dominance Principle in section 11. That section is about 1500 words long (so a 5-10 minute read). People familiar with the shutdown problem can read 'The idea in a nutshell' and then read from section 11 onwards. Here's a PDF version of this post. For those who like videos, this talk covers much of the same ground as this post.[1] The idea in a nutshell Here's the IPP in a nutshell: Create agents that lack a preference between every pair of different-length trajectories (that is: every pair of trajectories in which shutdown occurs after different lengths of time). (More) …because such agents won't pay costs to shift probability mass between different-length trajectories, and so won't pay costs to prevent or cause shutdown. (More) …and we humans can ensure that preventing or causing shutdown is always at least a little bit costly for these agents (e.g. in terms of resources), so these agents won't try to prevent or cause shutdown. (More) And here's an idea for training agents to lack a preference between every pair of different-length trajectories: Make one change to an otherwise-thoroughly-prosaic setup for training advanced AI: give agents lower reward for repeatedly choosing same-length trajectories. (More) This change incentivises agents to choose stochastically between different-length trajectories. …and stochastic choosing between different-length trajectories indicates a lack of preference between different-length trajectories. In using this method to train agents to satisfy the IPP, we largely circumvent the problems of reward misspecification, goal misgeneralization, and deceptive alignment. (More) Summary of this post I explain and motivate the shutdown problem: the problem of creating artificial agents that (1) shut down when a shutdown button is pressed, (2) don't try to prevent or cause the pressing of the shutdown button, and (3) otherwise pursue goals competently. (More) I present a simple theorem that formalises the problem and use the theorem to identify my proposed solution: creating agents with incomplete preferences. (More) Specifically, I propose that we create agents that lack a preference between every pair of different-length trajectories (that is: every pair of trajectories in which shutdown occurs after different lengths of time). (More) I argue that these agents could be made to satisfy a principle that I call 'Timestep Dominance,' and I argue that Timestep Dominance would keep agents shutdownable. (More) I suggest a way to train advanced agents to lack preferences between different-length trajectories and to satisfy Timestep Dominance. (More) I argue that this training method lets us largely circumvent the problems of reward misspecification, goal misgeneralization, and deceptive alignment. (More) I end with some limitations of the proposal and a list of issues still to address. (More) 1. Introduction AI labs are endowing artificial agents with tools like web-browsing abilities, robot limbs, and text-channels for communicating with humans. These labs are also training agents to pursue goals in the wider world. That requires agents exhibiting some understanding of the wider world, and agents with this understanding could use their tools to prevent us humans from shutting them down. These agents could make promises or threats, copy themselves to new servers, hide their bad behaviour, block our access to their power-source, and many other things bes...
LW - The Byronic Hero Always Loses by Cole Wyeth
昨日
LW - The Byronic Hero Always Loses by Cole Wyeth
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Byronic Hero Always Loses, published by Cole Wyeth on February 23, 2024 on LessWrong. Why do we root for the antiheroes? Walter White, Light Yagami, Lucifer Morningstar, Doctor Frankenstein... It seems to be natural to root for the bad guys, and perhaps even more natural for rationalists. "I must confess I have always had some sympathy with villains. Heroism makes fine entertainment but sooner or later someone has to get things done." - Bayaz, Joe Abercrombie's First Law Trilogy. I think this is the reason, at least for me. Villains (particularly the most beloved villains) are agentic. They know what they want and they attempt to obtain it through their own actions. Sometimes those actions are even reasonable. This is notable because it's usually untrue of heroes. Heroes tend to be called to adventure, accepting a goal thrust upon them by circumstances or a bearded stranger or both. Then they pursue it, but often alongside a handful of other eclectic goals which cause internal conflict when they can't all be satisfied (getting distracted rescuing some civilians when the fate of the world is at stake). There's also something fairly un-agentic about following an absolute "honor" code, which is much more common among heroes than villains. Finally, heroes have teams of friends instead of minions, and are often forced to go out of their way to help their friends. A hero's friendships might even change his terminal values! So, why do we often find villains so fascinating? I think it's the same reason that rationalists don't run the world. Being highly agentic just isn't easy in real life, because it's inherently individualistic. Individuals tend to be outcompeted by large organizations. Large organizations are approximately agentic, but the people making up a large organization don't necessarily have to be. In fact, it might be better if they aren't! The military seems to optimize much more strongly for loyalty and discipline than agency - in fact, an army of agents with diverse goals seems tricky to control, since most will instrumentally value their own survival (though an army of agents with one cohesive goal may be even more formidable). In industry, principles of comparative advantage can imply that it is best for each employee to become highly specialized, which seems opposed to developing their agency. A more capable agent might generalize (building the type of skill set that a startup found might need, including "executive nature" as in Competent Elites). Though agentic employees can create a lot of value, I think it is typically more cost effective to create additional cogs in the machine than it is to increase agency. I think this is also why politicians are not very clever (epistemic status: I don't know any politicians). The things we value in a leader include costly trustworthiness signals (see https://www.elephantinthebrain.com/). In fact, the risk of an actively corrupt leader may be great enough that some would prefer a candidate with actively irrational beliefs (say, expecting moral judgement in the afterlife). I would almost go so far as to say that the idea of becoming a Machiavellian scheming mastermind and changing the world (for better or worse) through sheer cleverness is a childish fantasy. Maybe that's not a bad thing; children might be naive, but at least they aren't bitter. Perhaps it's just my American upbringing, but I think I want to live in a world where agents can get what they want, even with the world set against them, if only they are clever and persistent enough. So when I read about another despicable villain sacrificing it all and throwing the natural order out of balance in the name of cursed immortality or to avenge their family or to grasp untold riches, I can't help but root for them a little. In a way, Milton's Lucifer is the ...
LW - Everything Wrong with Roko's Claims about an Engineered Pandemic by EZ97
昨日
LW - Everything Wrong with Roko's Claims about an Engineered Pandemic by EZ97
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Everything Wrong with Roko's Claims about an Engineered Pandemic, published by EZ97 on February 23, 2024 on LessWrong. Premises Here I tackle some bold claims made by Roko in three separate posts about SARS-CoV-2 being an engineered virus (Brute Force Manufactured Consensus is Hiding the Crime of the Century, The Math of Suspicious Coincidences, and A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is). A first rebuttal to Roko's post is already out, and since I have been preparing this post for some time, certain arguments are going to be similar. The purpose of this post is not to incontrovertibly demonstrate that SARS-CoV-2 was of zoonotic origin, as I myself believe a laboratory leak to be somewhat plausible, but rather that the degree of Roko's confidence in the engineered-pandemic belief is terribly overstated in light of the presented evidence. I believe that the RootClaim debate is a great way to familiarise oneself with core arguments from all sides, it is exceedingly comprehensive and up to date. I also found Wikipedia articles on the lab-leak hypothesis and adjacent topics to be pretty informative. I definitely drew inspiration from many comments to the main post, if you recognise a comment in this post please let me know and I will link it. A Set of Explainable Coincidences Roko asserts that there the odds of zoonotic spillover are 1 in 80,000,000. What are the chances that the bat coronavirus which caused a once in a century pandemic managed to navigate its way all the way from a Yunnan cave and home in on the city with the lab having the top Google hits for "Coronavirus China" and also the location of China's first BSL-4 lab? Well, that would be approximately 1 in 200, since that is the fraction of China's population in Wuhan. 'One-in-a-century' pandemic The '1 in 200' figure is restated here as Coincidence of Location: Wuhan is a particularly special place in China for studying covid-19; the WIV group was both the most important, most highly-cited group before 2020, and the only group that was doing GoF on bat sarbecoronaviruses as far as I know. Wuhan is about 0.5% of China's population. It's a suspicious coincidence that a viral pandemic would occur in the same city as the most prominent group that studies it. First of all, a global pandemic is much more likely to start in a large city with high internal and external traffic, most of which are bound to have research centres for virology research, especially if one is referring to a fast-growing megacity of 11 million inhabitants. Secondly, a zoonotic spillover requires frequent and direct human-animal contact, and wet markets are strong candidates for this. The cities with highest presence of wet markets with live animals per-capita are located in large Chinese cities in the centre-south (see the section 'From Yunnan to Wuhan' for more on this). Roko writes that (italic added) Coincidence of timing: several things happened that presaged the emergence of covid-19. In December 2017, the US government lifted a ban on risky pathogen research, and in mid-2018 the Ecohealth group started planning how to make covid in the DEFUSE proposal. A natural spillover event could have happened at any time over either the last, say, 40 years or (probably) the next 40 years, though likely not much before that due to changing patterns of movement (I need help on exactly how wide this time interval is). As explained throughout this post, global pandemics require a specific set of simultaneous circumstances, that is why most natural spillovers end up not being pandemics: because it is a rare combination of factors. In China alone, natural spillovers of various kind take place quite literally all the time, which should lead to a much higher prior probability of zoonosis. A five-...
LW - Gemini Has a Problem by Zvi
昨日
LW - Gemini Has a Problem by Zvi
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Gemini Has a Problem, published by Zvi on February 23, 2024 on LessWrong. Google's Gemini 1.5 is impressive and I am excited by its huge context window. I continue to default to Gemini Advanced as my default AI for everyday use when the large context window is not relevant. However, while it does not much interfere with what I want to use Gemini for, there is a big problem with Gemini Advanced that has come to everyone's attention. Gemini comes with an image generator. Until today it would, upon request, create pictures of humans. On Tuesday evening, some people noticed, or decided to more loudly mention, that the humans it created might be rather different than humans you requested… Joscha Bach: 17th Century was wild. [prompt was] 'please draw a portrait of a famous physicist of the 17th century.' Kirby: i got similar results. when I went further and had it tell me who the most famous 17th century physicist was, it hummed and hawed and then told me newton. and then this happened: This is not an isolated problem. It fully generalizes: Once the issue came to people's attention, the examples came fast and furious. Among other things: Here we have it showing you the founders of Google. Or a pope. Or a 1930s German dictator. Or hell, a 'happy man.' And another example that also raises other questions, were the founding fathers perhaps time-traveling comic book superheroes? The problem is not limited to historical scenarios. Nor do the examples involve prompt engineering, trying multiple times, or any kind of gotcha. This is what the model would repeatedly and reliably do, and users were unable to persuade the model to change its mind. Nate Silver: OK I assumed people were exaggerating with this stuff but here's the first image request I tried with Gemini. Gemini also flat out obviously lies to you about why it refuses certain requests. If you are going to say you cannot do something, either do not explain (as Gemini in other contexts refuses to do so) or tell me how you really feel, or at least I demand a plausible lie: It is pretty obvious what it is the model has been instructed to do and not to do. Owen Benjamin: The only way to get AI to show white families is to ask it to show stereotypically black activities. … For the record it was a dude in my comment section on my last post who cracked this code. This also extends into political issues that have nothing to do with diversity. The Internet Reacts The internet, as one would expect, did not take kindly to this. That included the usual suspects. It also included many people who think such concerns are typically overblown or who are loathe to poke such bears, such as Ben Thompson, who found this incident to be a 'this time you've gone too far' or emperor has clothes moment. St. Ratej (Google AR/VR, hey ship a headset soon please, thanks): I've never been so embarrassed to work for a company. Jeffrey Emanuel: You're going to get in trouble from HR if they know who you are… no one is allowed to question this stuff. Complete clown show. St. Ratej: Worth it. Ben Thompson (gated) spells it out as well, and has had enough: Ben Thompson: Stepping back, I don't, as a rule, want to wade into politics, and definitely not into culture war issues. At some point, though, you just have to state plainly that this is ridiculous. Google specifically, and tech companies broadly, have long been sensitive to accusations of bias; that has extended to image generation, and I can understand the sentiment in terms of depicting theoretical scenarios. At the same time, many of these images are about actual history; I'm reminded of George Orwell in 1984: George Orwell (from 1984): Every record has been destroyed or falsified, every book has been rewritten, every picture has been repainted, every statue and street and building has been renamed, ...
LW - AI #52: Oops by Zvi
昨日
LW - AI #52: Oops by Zvi
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #52: Oops, published by Zvi on February 23, 2024 on LessWrong. We were treated to technical marvels this week. At Google, they announced Gemini Pro 1.5, with a million token context window within which it has excellent recall, using mixture of experts to get Gemini Advanced level performance (e.g. GPT-4 level) out of Gemini Pro levels of compute. This is a big deal, and I think people are sleeping on it. Also they released new small open weights models that look to be state of the art. At OpenAI, they announced Sora, a new text-to-video model that is a large leap from the previous state of the art. I continue to be a skeptic on the mundane utility of video models relative to other AI use cases, and think they still have a long way to go, but this was both technically impressive and super cool. Also, in both places, mistakes were made. At OpenAI, ChatGPT briefly lost its damn mind. For a day, faced with record traffic, the model would degenerate into nonsense. It was annoying, and a warning about putting our trust in such systems and the things that can go wrong, but in this particular context it was weird and beautiful and also hilarious. This has now been fixed. At Google, people noticed that Gemini Has a Problem. In particular, its image generator was making some highly systematic errors and flagrantly disregarding user requests, also lying about it to users, and once it got people's attention things kept looking worse and worse. Google has, to their credit, responded by disabling entirely the ability of their image model to output people until they can find a fix. I hope both serve as important warnings, and allow us to fix problems. Much better to face such issues now, when the stakes are low. Table of Contents Covered separately: Gemini Has a Problem, Sora What, and Gemini 1.5 Pro. Introduction. We've got some good news, and some bad news. Table of Contents. Language Models Offer Mundane Utility. Probable probabilities? Language Models Don't Offer Mundane Utility. Air Canada finds out. Call me Gemma Now. Google offers new state of the art tiny open weight models. Google Offerings Keep Coming and Changing Names. What a deal. GPT-4 Goes Crazy. But it's feeling much better now. GPT-4 Real This Time. Offer feedback on GPTs, see their profiles. Fun With Image Generation. Image generation for journal articles. Deepfaketown and Botpocalypse Soon. Several approaches to impersonation risks. Selling Your Chatbot Data. I don't really know what you were expecting. Selling Your Training Data. I still don't really know what you were expecting. They Took Our Jobs. There is a third option. Get Involved. Apart Research is hiring. Introducing. Groq, Lindy, Podcaster Copilot, potentially Magic and Altera. In Other AI News. Altman looks to move his chip plans forward. Quiet Speculations. Arguing over slow versus fast takeoff during takeoff. The Quest for Sane Regulations. There will be many bills along the way. The Week in Audio. I'm back on the Cognitive Revolution. The Original Butlerian Jihad. What was Dune a cautionary tale against again? Rhetorical Innovation. Another open letter, another trillion dollars. Ho hum. Public Service Announcement. Fentanyl, both literally and as metaphor. People Are Worried About AI Killing Everyone. There is a pattern to who. Other People Are Not As Worried About AI Killing Everyone. Sure, why not. The Lighter Side. There is not enough information to solve the problem. Language Models Offer Mundane Utility Steven Johnson strongly endorses NotebookLM, offers YouTube tutorial. This is definitely one of those 'I need to try using this more and it's weird I don't find excuses' situations. Automatically email everyone to tell them to remove your email address from their database. Patrick McKenzie: Interestingly, one of the first denial of service vi...
LW - Research Post: Tasks That Language Models Don't Learn by Bruce W. Lee
昨日
LW - Research Post: Tasks That Language Models Don't Learn by Bruce W. Lee
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Research Post: Tasks That Language Models Don't Learn, published by Bruce W. Lee on February 23, 2024 on LessWrong. Abstract We argue that there are certain properties of language that our current large language models (LLMs) don't learn. We present an empirical investigation of visual-auditory properties of language through a series of tasks, termed H-Test. This benchmark highlights a fundamental gap between human linguistic comprehension, which naturally integrates sensory experiences, and the sensory-deprived processing capabilities of LLMs. In support of our hypothesis, 1. deliberate reasoning (Chain-of-Thought), 2. few-shot examples, or 3. stronger LLM from the same model family (LLaMA 2 13B -> LLaMA 2 70B) do not trivially bring improvements in H-Test performance. Therefore, we make a particular connection to the philosophical case of Mary, who learns about the world in a sensory-deprived environment. Our experiments show that some of the strongest proprietary LLMs stay near random chance baseline accuracy of 50%, highlighting the limitations of knowledge acquired in the absence of sensory experience. Key Findings on H-Test 1. Insignificant Intra-Family Improvements. A stronger model in the same model family often does not bring meaningful improvement in H-Test performance. 2. Number of Examples Has Minimal Impact. The number of examples given neither increases nor decreases performance significantly, strongly hinting that the LLM is simply not learning from H-Test few-shot examples. 3. Deliberate Reasoning (CoT) Often Decreases Performance. If LLMs benefit from such logical, step-by-step semantic reasoning on the H-Test, this can also imply that H-Test is fundamentally solvable by developing stronger language-only models. But CoT decreases performances in general. 4. Training with more orthography-specific language data does not improve H-Test. We produced 1000 training instances per task in H-Test and fine-tuned gpt-3.5-turbo-0613 ten different times accordingly. After training for three epochs on each task, we evaluated them on H-Test at k = 50 and observed that no significant performance improvement was achieved. 5. Multi-modality does not automatically improve H-Test performance. At the time of writing, LLaVA V1.6 34B is the strongest open-source multi-modal model available. Despite the addition of visual modality, we observe that simply incorporating visual data into the training does not result in a straightforward improvement in H-Test performance. 6 (Important). But the H-Test is solvable, we just don't know how. In our paper, we have reported the seemingly unexplainable (jumping) performance improvement on H-Test from GPT-3.5 to GPT-4. This result is important as it shows that H-Test is indeed solvable (by a GPT-4-level system), but not through conventionally discussed language-only modeling techniques. Acknowledgments and Links I was inspired to do this research due to examples cases given in @Owain_Evans track application for the Constellation Fellowship. This research is not financially supported by a university lab, and Walnut Research is an independent research organization that runs on personal funding from myself and a few of my friends. ArXiv: https://arxiv.org/abs/2402.11349 GitHub: https://github.com/brucewlee/H-Test Twitter: https://x.com/BruceWLee1/status/1760736653747081681?s=20 Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
LW - Sora What by Zvi
昨日
LW - Sora What by Zvi
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Sora What, published by Zvi on February 23, 2024 on LessWrong. Hours after Google announced Gemini 1.5, OpenAI announced their new video generation model Sora. Its outputs look damn impressive. How Sora Works How does it work? There is a technical report. Mostly it seems like OpenAI did standard OpenAI things, meaning they fed in tons of data, used lots of compute, and pressed the scaling button super hard. The innovations they are willing to talk about seem to be things like 'do not crop the videos into a standard size.' That does not mean there are not important other innovations. I presume that there are. They simply are not talking about the other improvements. We should not underestimate the value of throwing in massively more compute and getting a lot of the fiddly details right. That has been the formula for some time now. Some people think that OpenAI was using a game engine to learn movement. Sherjil Ozair points out that this is silly, that movement is learned easily. The less silly speculation is that game engine outputs may have been in the training data. Jim Fan thinks this is likely the case, and calls the result a 'data-driven physics engine.' Raphael Molière thinks this is likely, but more research is needed. Brett Goldstein here digs into what it means that Sora works via 'patches' that combine to form the requested scene. Gary Marcus keeps noting how the model gets physics wrong in various places, and, well, yes, we all know, please cut it out with the Stop Having Fun. Yishan points out that humans also work mostly on 'folk physics.' Most of the time humans are not 'doing logic' they are vibing and using heuristics. I presume our dreams, if mapped to videos, would if anything look far less realistic than Sora. Yann LeCun, who only a few days previous said that video like Sora produces was not something we knew how to do, doubled down with the ship to say that none of this means the models 'understand the physical world,' and of course his approach is better because it does. Why update? Is all of this technically impressive? Sora Is Technically Impressive Yes, Sora is definitely technically impressive. It was not, however, unexpected. Sam Altman: we'd like to show you what sora can do, please reply with captions for videos you'd like to see and we'll start making some! Eliezer Yudkowsky: 6 months left on this timer. Eliezer Yudkowsky (August 26, 2022): In 2-4 years, if we're still alive, anytime you see a video this beautiful, your first thought will be to wonder whether it's real or if the AI's prompt was "beautiful video of 15 different moth species flapping their wings, professional photography, 8k, trending on Twitter". Roko (other thread): I don't really understand why anyone is freaking out over Sora. This is entirely to be expected given the existence of generative image models plus incrementally more hardware and engineering effort. It's also obviously not dangerous (in a "take over the world" sense). Eliezer Yudkowsky: This is of course my own take (what with having explicitly predicted this). But I do think you want to hold out a space for others to say, "Well *I* didn't predict it, and now I've updated." Altman's account spent much of last Thursday making videos for people's requests, although not so many that they couldn't cherry pick the good ones. As usual, there are failures that look stupid, mistakes 'a person would never make' and all that. And there are flashes of absolute brilliance. How impressive? There are disputes. Tom Warren: this could be the "holy shit" moment of AI. OpenAI has just announced Sora, its text-to-video AI model. This video isn't real, it's based on a prompt of "a cat waking up its sleeping owner demanding breakfast…" Daniel Eth: This isn't impressive. The owner doesn't wake up, so the AI clearly didn't understa...
LW - Contra Ngo et al. "Every 'Every Bay Area House Party' Bay Area House Party" by Ricki Heicklen
昨日
LW - Contra Ngo et al. "Every 'Every Bay Area House Party' Bay Area House Party" by Ricki Heicklen
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Contra Ngo et al. "Every 'Every Bay Area House Party' Bay Area House Party", published by Ricki Heicklen on February 23, 2024 on LessWrong. With thanks to Scott Alexander for the inspiration, Jeffrey Ladish, Philip Parker, Avital Morris, and Drake Thomas for masterful cohosting, and Richard Ngo for his investigative journalism. Last summer, I threw an Every Bay Area House Party themed party. I don't live in the Bay, but I was there for a construction-work-slash-webforum-moderation-and-UI-design-slash-grantmaking gig, so I took the opportunity to impose myself on the ever generous Jeffrey Ladish and host a party in his home. Fortunately, the inside of his house is already optimized to look like a parody of a Bay Area house party house, so not much extra decorating was needed, but when has that ever stopped me? Richard Ngo recently covered the event, with only very minor embellishments. I've heard rumors that some people are doubting whether the party described truly happened, so I'd like to set the record straight. Thus, this is part linkpost, part exegesis, and part shameless promotion of my events for potential future venue-lenders. The party had 60 attendees, at least according to the Manifold Market on the topic. Upon arrival, partygoers put on tags with their name, professional title, and LessWrong karma. Attendees were also instructed to put on a wristband that successfully communicated their flirting policies. I took the wristband for people who glomarize about their flirting preferences; Richard took the wristband for those who flirt with all and only those who don't flirt with themselves. Richard writes: You scan around the living room, trying to figure out who to talk to first. The host is sitting on the sofa, with two boxes attached to the front of her shirt. One is filled with money, the other empty. A guy next to her is surreptitiously one-boxing, but she presciently slaps his hand away without even looking. This is defamation. The second box wasn't (necessarily) empty, and Richard certainly never got the opportunity to look inside it. You might be wondering what was in the box. Unfortunately for you, I glomarize not only about my flirting policies but also about my box contents. He is correct, though, that I managed to fend off all the surreptitious one-boxers, with the exception of my girlfriend Avital. She still doesn't know the contents - I would never date someone irresponsible enough to let unknown entities out of a box. The party was PYOL (provide your own liquidity), but we did offer two punches: one for "Contextualizers," and one for "Decouplers or Homophobes". Avital and I drank the punch for "Decouplers or Homophobes." We're still coupled, so you can come to your own conclusions about how homophobic we must be. My favorite part of the night happened when a circle formed around a Jewish Orthodox Rabbi friend of mine who had never heard of Rationality or Effective Altruism. Everyone at the party was eager to give him context. I joined the circle as they were discussing expanding moral circles, and the relative weight of animal lives and future people. "Eliezer doesn't even think cows are sentient," one attendee was saying. "But shrimp are!" another interrupted, causing the group to crack up. "What?" my Rabbi friend said. "Okay, back up, how much Peter Singer have you read?" another attendee said. "Like, have you read Animal Liberation and The Expanding Circle, or just Famine, Affluence, and Morality?" Avital and I tried to explain to them that our friend had not heard of any of the people they were naming, but they didn't seem to understand. Avital turned to me and said "I guess when you've read The Sequences too many times you forget about inferential distance." Eventually, my Rabbi friend said "Okay, so what I'm hearing is: you're expected to t...
LW - Do sparse autoencoders find "true features"? by Demian Till
2日前
LW - Do sparse autoencoders find "true features"? by Demian Till
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Do sparse autoencoders find "true features"?, published by Demian Till on February 22, 2024 on LessWrong. In this post I'll discuss an apparent limitation of sparse autoencoders (SAEs) in their current formulation as they are applied to discovering the latent features within AI models such as transformer-based LLMs. In brief, I'll cover the following: I'll argue that the L1 regularisation used to promote sparsity when training SAEs may cause neurons in the sparse layer to learn to represent common combinations of features rather than the individual features that we want them to discover As well as making it more difficult to understand what the actual latent features are, I'll also argue that this limitation may result in some less common latent features not being discovered at all, not even within combinations I'll then explain why I think that the phenomenon of feature splitting observed in Anthropic's SAE paper appears to demonstrate that this limitation does indeed have a large impact on the features discovered by SAEs Finally I'll propose an approach for overcoming this limitation and discuss how we can test whether it really brings us closer to finding the real latent features Rough definition of "true features" We intend for SAEs to discover the "true features" (a term I'm borrowing from Anthropic's SAE paper) used by the target model e.g. a transformer-based LLM. There isn't a universally accepted definition of what "true features" are, but for now I'll use the term somewhat loosely to refer to something like: linear directions in an activation space at a hidden layer within a target model which encode some reasonably monosemantic quantity such as the model's "confidence" in some concept being in play they should play a causal role in the functioning of the target model. So for example if we were to activate or deactivate the feature while the target model is processing a given input sequence then we should expect the outputs to change accordingly in some reasonably understandable way they should be in their most atomic form, so that e.g an arbitrary linear combination of two "true feature" directions is not necessarily itself a "true feature" direction even though it may satisfy the previous criteria There may be other ways of thinking about features but this should give us enough to work with for our current purposes. Why SAEs are incentivised to discover combinations of features rather than individual features Consider a toy setup where one of the hidden layers in the target model has 3 "true features" represented by the following directions in its activation space: Additionally, suppose that feature 1 and feature 2 occur far more frequently than feature 3, and that all features can potentially co-occur in a given activation vector. For the sake of simplicity let's also suppose for now that when features 1 & 2 occur together they tend to both activate with some roughly fixed proportions. For example, an activation vector in which both features 1 and 2 are present (but not feature 3) might look like the following: Now suppose we train an SAE with 3 neurons in the sparse layer on activation vectors from this hidden layer such as the one above. The desirable outcome is that each of the 3 neurons in the sparse layer learns one of the 3 "true features". If this happens then the directions learnt by SAE would mirror the directions of the "true features" in the target model, looking something like: However depending on the respective frequencies of feature 3 vs features 1 & 2, as well as the value of the L1 regularisation weight, I will argue shortly that what may happen is that two of the neurons learn to detect when each of features 1 & 2 respectively occur by themselves, while the third neuron learns to detect when they both occur together. In this case the di...
EA - From salt intake reduction to labor migration: Announcing top ideas for the AIM 2024 CE Incubation Program by CE
2日前
EA - From salt intake reduction to labor migration: Announcing top ideas for the AIM 2024 CE Incubation Program by CE
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: From salt intake reduction to labor migration: Announcing top ideas for the AIM 2024 CE Incubation Program, published by CE on February 22, 2024 on The Effective Altruism Forum. TLDR: In this post, we announce our top four charity ideas to launch through our August 12-October 4, 2024 Incubation Program. They are the result of months of work by our research team, who selected them through a five-stage research process. We pick interventions that exceed ambitious cost-effectiveness bars, have a high quality of evidence, minimal failure modes, and high expected value. We're also announcing cause areas we'll investigate for the February-March 2025 IP. We're seeking people to launch these ideas through our next Incubation Program. No particular previous experience is necessary - if you could plausibly see yourself excited to launch one of these charities, we encourage you to apply. The deadline for applications is April 14, 2024. You can apply to both August-October 2024 or February-March 2025 programs via the same link below: In the Incubation Program, we provide two months of cost-covered training, stipends (£1900/month during and for up to two months after the program), seed funding up to $200,000, operational support in the initial months, co-working space at our CE office in London, ongoing mentorship, and access to a community of alumni, funders, and experts. Learn more on our CE Incubation Program page. One sentence summaries Advocacy for salt intake reduction An organization seeking to convince governments and the food industry to lower the content of salt in food by setting sodium limits and reformulating high-sodium foods, thereby improving cardiovascular health. Facilitating international labor migration via a digital platform An organization that would facilitate the international migration of workers from low- and middle-income countries using a transparent digital platform paired with low-cost personalized support. Ceramic filters for improving water quality An organization focused on reducing the incidence of diarrhea and other waterborne illnesses by providing free ceramic water filters to families without access to clean drinking water. Participatory Learning and Action (PLA) groups for maternal and newborn health An organization focused on improving newborn and maternal health in rural villages by training facilitators and running PLA groups - a specific type of facilitated self-help group. One-paragraph Summaries Advocacy for salt intake reduction High salt consumption contributes to poor cardiovascular health. Worldwide, cardiovascular diseases are the leading cause of death and are among the top ten contributors to years lived with disabilities. There is good evidence that reducing the amount of sodium people consume in their diets can reduce the risk of cardiovascular problems. Therefore, several countries have successfully reduced the sodium intake of their population, for example by setting sodium limits, which led to the food industry reformulating certain high-salt products. We think that an organization advocating and assisting in implementing these policies could cost-effectively improve the health of millions. Facilitating international labor migration via a digital platform Migrating abroad for work can bring huge financial benefits to people experiencing poverty. Millions of people every year try to tap into these benefits by applying for temporary jobs in higher-income countries. However, the market for matching candidates with jobs is often highly inefficient and riddled with misinformation, putting candidates at financial and personal risk. In many countries, it can cost candidates several years' worth of salaries to secure a job abroad. Fraud is also highly prevalent, leading to candidates often failing to migrate and instead ending up i...
LW - The One and a Half Gemini by Zvi
2日前
LW - The One and a Half Gemini by Zvi
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The One and a Half Gemini, published by Zvi on February 22, 2024 on LessWrong. Previously: I hit send on The Third Gemini, and within half an hour DeepMind announced Gemini 1.5. So this covers Gemini 1.5. One million tokens, and we are promised overall Gemini Advanced or GPT-4 levels of performance on Gemini Pro levels of compute. This post does not cover the issues with Gemini's image generation, and what it is and is not willing to generate. I am on top of that situation and will get to it soon. One Million Tokens Our teams continue pushing the frontiers of our latest models with safety at the core. They are making rapid progress. In fact, we're ready to introduce the next generation: Gemini 1.5. It shows dramatic improvements across a number of dimensions and 1.5 Pro achieves comparable quality to 1.0 Ultra, while using less compute. It is truly bizarre to launch Gemini Advanced as a paid service, and then about a week later announce the new Gemini Pro 1.5 is now about as good as Gemini Advanced. Yes, actually, I do feel the acceleration, hot damn. And that's not all! This new generation also delivers a breakthrough in long-context understanding. We've been able to significantly increase the amount of information our models can process - running up to 1 million tokens consistently, achieving the longest context window of any large-scale foundation model yet. One million is a lot of tokens. That covers every individual document I have ever asked an LLM to examine. That is enough to cover my entire set of AI columns for the entire year, in case I ever need to look something up, presumably Google's NotebookLM is The Way to do that. A potential future 10 million would be even more. Soon Gemini will be able to watch a one hour video or read 700k words, whereas right now if I use the web interface of Gemini Advanced interface all I can upload is a photo. The standard will be to give people 128k tokens to start, then you can pay for more than that. A million tokens is not cheap inference, even for Google. Oriol Vinyals (VP of R&D DeepMind): Gemini 1.5 has arrived. Pro 1.5 with 1M tokens available as an experimental feature via AI Studio and Vertex AI in private preview. Then there's this: In our research, we tested Gemini 1.5 on up to 2M tokens for audio, 2.8M tokens for video, and 10M tokens for text. From Shannon's 1950s bi-gram models (2 tokens), and after being mesmerized by LSTMs many years ago able to model 200 tokens, it feels almost impossible that I would be talking about hundreds of thousands of tokens in context length, let alone millions. Jeff Dean (Chief Scientist, Google DeepMind): Multineedle in haystack test: We also created a generalized version of the needle in a haystack test, where the model must retrieve 100 different needles hidden in the context window. For this, we see that Gemini 1.5 Pro's performance is above that of GPT-4 Turbo at small context lengths and remains relatively steady across the entire 1M context window, while the GPT-4 Turbo model drops off more quickly (and cannot go past 128k tokens). Guido Appenzeller (responding to similar post): Is this really done with a monolithic model? For a 10M token window, input state would be many Gigabytes. Seems crazy expensive to run on today's hardware. Sholto Douglas (DeepMind): It would honestly have been difficult to do at decent latency without TPUs (and their interconnect) They're an under appreciated but critical piece of this story Here are their head-to-head results with themselves: Here is the technical report. There is no need to read it, all of this is straightforward. Their safety section says 'we followed our procedure' and offers no additional details on methodology. On safety performance, their tests did not seem to offer much insight, scores were similar to Gemini Pro 1.0. Mixture...
EA - Upcoming EA conferences in 2024 by OllieBase
2日前
EA - Upcoming EA conferences in 2024 by OllieBase
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Upcoming EA conferences in 2024, published by OllieBase on February 22, 2024 on The Effective Altruism Forum. In an unsurprising move, the Centre for Effective Altruism will be organising and supporting conferences for the EA community all over the world in 2024, including three new EAGx locations: Copenhagen, Toronto and Austin. We currently have the following events scheduled: EA Global EA Global: London | (May 31-June 2) | Intercontinental London (the O2) - applications close 19 May EA Global: Boston | (November 1-3) | Hynes Convention Center - applications close 20 October EAGx EAGxAustin | (April 13-14) | University of Texas, Austin - applications close 31 March EAGxNordics | (April 26-28) | CPH Conference, Copenhagen - applications close 7 April EAGxUtrecht | (July 5-7) | Jaarbeurs, Utrecht EAGxToronto | (August, provisional) EAGxBerkeley | (September, provisional) EAGxBerlin | (September 13-15) | Urania, Berlin EAGxAustralia | (November) | Sydney We also hope to announce an EAGxLondon for early April very soon. A university venue was tentatively booked for late March, but the venue asked to reschedule. We're in the process of finalising a new date. We also expect to announce more events throughout the year. Applications for EAG London, EAG Boston, EAGxNordics and EAGxAustin are open. Applications for EAGxLondon will open as soon as the date is confirmed. We expect applications for the other conferences to open approximately 3 months before the event. Please go to the event page links above to apply. If you'd like to add EAG(x) events directly to your Google Calendar, use this link. Some notes on these conferences: EA Globals are run in-house by the CEA events team, whereas EAGx conferences are organised independently by members of the EA community with financial support and mentoring from CEA. EA Global conferences have a high bar for admission and are for people who are very familiar with EA and are taking significant actions (e.g. full-time work or study) based on EA ideas. Admissions for EAGx conferences are processed independently by the EAGx conference organizers. These events are primarily for those who are newer to EA and interested in getting more involved. Please apply to all conferences you wish to attend once applications open - we would rather get too many applications for some conferences and recommend that applicants attend a different one than miss out on potential applicants to a conference. Travel support funds for events this year are limited (though will vary by event), and we can only accommodate a small number of requests. If you do not end up receiving travel support, this is likely the result of limited funds, rather than an evaluation of your potential for impact. When planning around an event, we recommend you act under the assumption that we will not be able to grant your travel funding request (unless it has already been approved). Find more info on our website. Feel free to email hello@eaglobal.org with any questions, or comment below. You can contact EAGx organisers using the format [location]@eaglobalx.org (e.g. austin@eaglobalx.org and nordics@eaglobalx.org). Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
LW - The Pareto Best and the Curse of Doom by Screwtape
2日前
LW - The Pareto Best and the Curse of Doom by Screwtape
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Pareto Best and the Curse of Doom, published by Screwtape on February 22, 2024 on LessWrong. I. Prerequisite reading: Being the (Pareto) Best in the World. A summary of Being the (Pareto) Best in the World: Being the world's best mathematician is hard. Being the world's best musician is hard. Being the world's best mathematician/musician is much easier, especially since there are multiple slots; an amazing mathematician who is also a competent musician, someone who is good at both, and a competent mathematician who is also an amazing musician can all find a niche. I like this concept, and have kept it in my back pocket ever since I read it. I have sometimes described myself as a software engineer who was competent at public speaking and project management. That particular overlapping skillset is, it turns out, fairly valuable. While I was attempting to become a better software engineer, I was also trying to add competence at corporate budgets and accounting to that skillset. These days I spend a lot of time talking to the kind of person who hangs out on LessWrong a lot or spends a lot of time going to Astral Codex Ten meetups. If ever I faced a problem that required a brilliant neuroscientist, or a gifted Haskell programmer, or a world leading expert in training honeybees, well, let's just say I know somebody. There are people out there who are exemplary at the thing they do. Sometimes they're not very good at other things though. While Being The (Pareto) Best in the World felt optimistic when I first read it, these days I regard it as a curse of doom upon the world, blighting otherwise promising areas of effort and endeavor. I look around at places where it feels like everyone is dropping the ball and see a blasted wasteland where nothing grows because nobody has the right combination of seemingly basic skills. II. Imagine a toy model where everyone has a hundred points to put into being good at things. (This is, to be clear, not just a toy model but an incorrect model. It's easy to look at your incoming university students and notice a strong inverse correlation between math and verbal SAT scores, forgetting that those get summed together during applications and anyone below a certain threshold probably has their application discarded. Still, let's use this model for the moment.) Leading talents in a field maybe put 75 points in their area. Why not 100? Because you need points in living your life. There's an archetype of the absent minded professor, someone who can understand a complex abstract subject but who shows up to give lectures having forgotten to put their shoes on or eat breakfast. Hitting 90 points in your field requires someone else to do a lot of the upkeep for you; many FAANG jobs provide food and other amenities, and I don't think it's entirely because it's a cheap perk. Politely, I know some FAANG engineers who I suspect would forget lunch and dinner if it was not conveniently provided for them. At sufficiently high levels of dedication, seemingly important related skills start to fall by the wayside. Many programmers are not good at documenting their code, writing or reading specifications, or estimating story points and timelines. Fiction authors vary wildly in their comfort with self-promotion, proofreading, and layout. That's what publishers and agents are for. There's a few indie musicians I enjoy whose mastery of sound mixing or recording technology is not the equal to their actual playing. You can spend 40 points on singing, 40 points on recording, and 20 points on living your life. At this point, you're giving up some noticeable quality somewhere. I'll arbitrarily draw a line at 50 points and say this is where so-called "professional" quality tends to hang out, the people you see do their thing and you think "man, they could make a livin...
LW - Dual Wielding Kindle Scribes by mesaoptimizer
3日前
LW - Dual Wielding Kindle Scribes by mesaoptimizer
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Dual Wielding Kindle Scribes, published by mesaoptimizer on February 21, 2024 on LessWrong. This is an informal post intended to describe a workflow / setup that I found very useful, so that others might consider adopting or experimenting with facets of it that they find useful. In August 2023, I was a part of MATS 4.0 and had begun learning the skill of deconfusion, with an aim of disentangling my conflicting intuitions between my belief that shard theory seemed to be at least directionally pointing at some issues with the MIRI model of AGI takeoff and alignment difficulty, and my belief that Nate Soares was obviously correct that reflection will break Alex Turner's diamond alignment scheme. A friend lent me his Kindle Scribe to try out as part of my workflow. I started using it for note-taking, and found it incredibly useful and bought it from him. A month later, I bought a second Kindle Scribe to add to my workflow. It has been about six months since, and I've sold both my Kindle Scribes. Here's why I found this workflow useful (and therefore why you might find it useful), and why I moved on from it. The Display The Kindle Scribe is a marvelous piece of hardware. With a 300 PPI e-ink 10.3 inch screen, reading books on it was a delight in comparison to any other device I've used to read content on. The stats I just mentioned matter: 300 PPI on a 10.3 inch display means the displayed text is incredibly crisp, almost indistinguishable from normal laptop and smartphone screens. This is not the case for most e-ink readers. E-ink screens seem to reduce eye strain by a non-trivial amount. I've looked into some studies, but the sample sizes and effect sizes were not enough to make me unilaterally recommend people switch to e-ink screens for reading. However, it does seem like the biggest benefit of using e-ink screens seems to be that you aren't staring into a display that is constantly shining light into your eyeballs, which is the equivalent of staring into a lightbulb. Anecdotally, it did seem like I was able to read and write for longer hours when I only used e-ink screens: I went from, about 8 to 10 hours a day (with some visceral eye fatigue symptoms like discomfort at the end of the day) to about 12 to 14 hours a day, without these symptoms, based on my informal tracking during September 2023. 10.3 inch screens (with a high PPI) just feel better to use in comparison to smaller (say, 6 to 7 inch screens) for reading. This seems to me to be due to a greater amount of text displayed on the screen at any given time, which seems to somehow limit the feeling of comprehensibility of the text. I assume this is somehow related to chunking of concepts in working memory, where if you have a part of a 'chunk' on one page, and another part on another page, you may have a subtle difficulty with comprehending what you are reading (if it is new to you), and the more the text you have in front of you, the more you can externalize the effort of comprehension. (I used a Kobo Libra 2 (7 inch e-ink screen) for a bit to compare how it felt to read on, to get this data.) Also, you can write notes in the Kindle Scribe. This was a big deal for me, since before this, I used to write notes on my laptop, and my laptop was a multi-purpose device. Sidenote: My current philosophy of note-taking is that I think 'on paper' using these notes, and don't usually refer to it later on. The aim is to augment my working memory with an external tool, and the way I write notes usually reflects this -- I either write down most of my relevant and conscious thoughts as I think them (organized as a sequence of trees, where each node is a string representing a 'thought'), or I usually write 'waypoints' for my thoughts, where each waypoint is a marker for a conclusion of a sequence / tree of thoughts, or an inte...
EA - In memory of Steven M. Wise by Tyner
3日前
EA - In memory of Steven M. Wise by Tyner
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: In memory of Steven M. Wise, published by Tyner on February 21, 2024 on The Effective Altruism Forum. LINK: https://everloved.com/life-of/steven-wise/obituary/ Renowned animal rights pioneer Steven M. Wise passed away on February 15th after a long illness. He was 73 years old. An innovative scholar and groundbreaking expert on animal law, Wise founded and served as president of the Nonhuman Rights Project (NhRP), the only nonprofit organization in the US dedicated solely to establishing legal rights for nonhuman animals. As the NhRP's lead attorney, he filed historic lawsuits demanding the right to liberty of captive chimpanzees and elephants, achieving widely recognized legal firsts for his clients. Most notably, under Wise's leadership the NhRP filed a habeas corpus petition on behalf of Happy, an elephant held alone in captivity at the Bronx Zoo. Happy's case, which historian Jill Lepore has called "the most important animal-rights case of the 21st-century," reached the New York Court of Appeals in 2022. The Court of Appeals then became the highest court of an English-speaking jurisdiction to hear arguments calling for a legal right for an animal. Although the Court ultimately denied Happy's petition, two judges wrote historic dissents refuting the idea that only humans can have rights. Under Wise's leadership, the NhRP also helped develop and pass the first animal rights law in the country in 2023-an ordinance that protects elephants' right to liberty. Wise said he decided to become a lawyer after developing a deep commitment to social justice as a result of his involvement in the anti-Vietnam War movement while an undergraduate at the College of William and Mary. He graduated from Boston University Law School in 1976 and began his legal career as a criminal defense lawyer. Several years later, Peter Singer's book Animal Liberation inspired Wise to become an animal protection lawyer. From 1985 to 1995, Wise was president of the Animal Legal Defense Fund. As Wise told The New York Times Magazine, his litigation work during this time led him to conclude that the rightlessness of animals was the fundamental barrier to humans vindicating animals' interests. This is because, under animal welfare laws, lawyers must make the case for how a human has been harmed by the animal's treatment or situation; as Wise elaborated in his writings and talks, legal injuries to animals do not matter in court because animals are unjustly considered legal "things" with no rights, legally equivalent to inanimate objects, their intrinsic interests essentially invisible to judges. In 1995, Wise launched the Nonhuman Rights Project to address this core issue facing all animals and their advocates. After more than a decade of preparation, the NhRP filed first-of-their-kind lawsuits in 2013, demanding rights for four captive chimpanzees in New York State. A year and a half later, two of the NhRP's clients became the first animals in legal history to have habeas corpus hearings to determine the lawfulness of their imprisonment. Wise was also a leading force in the development of animal law as a distinct academic curriculum, teaching the first-ever animal law course offered at Harvard University in 2000. He remained committed to educating the next generation of animal rights lawyers throughout his career, teaching animal rights jurisprudence at law schools around the world, including Stanford Law School, the University of Miami Law School, St. Thomas University Law School, John Marshall Law School, Lewis and Clark Law School, Vermont Law School, Tel Aviv University, and the Autonomous University of Barcelona. Wise is the author of four books: Rattling the Cage: Toward Legal Rights for Animals (2000); Drawing the Line: Science and the Case for Animal Rights (2002); Though the Heavens May Fall: T...