No announcement yet.

Punditry and prediction

  • Filter
  • Time
  • Show
Clear All
new posts

  • Punditry and prediction

    I want to draw attention to some scientific work on political punditry. The research tested several hundred political pundits having them make long term predictions on matters closely and not so closely related to their fields of interest. They could choose three options based broadly on probabilities scales.

    The psychologist Philip Kaufmann has concluded they performed worse than "dart throwing monkeys" and would have been better off just assigning equal probabilty to each option to all predictions The research found that performance was barley better than random, and that those that were the most famous or most specialised performed the poorest.

    This has led to the development of the Good Judgement Project and has significant consequences in both economics and geopolitics.

    These findings were reported widely in the media and came to the attention of Intelligence Advanced Research Projects Activity (IARPA) inside the United States intelligence community—a fact that was partly responsible for the 2011 launch of a four-year geopolitical forecasting tournament that engaged tens of thousands of forecasters and drew over one million forecasts across roughly 500 questions of relevance to U.S. national security, broadly defined.

    Since 2011, Tetlock and his wife/research partner Barbara Mellers have been co-leaders of the Good Judgment Project (GJP), a research collaborative that emerged as the winner of the IARPA tournament.[3] The original aim of the tournament was to improve geo-political and geo-economic forecasting. Illustrative questions include “What is the chance that a member will withdraw from the European Union by a target date?” or “What is the likelihood of naval clashes claiming over 10 lives in the East China Sea?” or “How likely is the head of state of Venezuela to resign by a target date?” The tournament challenged GJP and its competitors at other academic institutions to come up with innovative methods of recruiting gifted forecasters, methods of training forecasters in basic principles of probabilistic reasoning, methods of forming teams that are more than the sum of their individual parts and methods of developing aggregation algorithms that most effectively distill the wisdom of the crowd.[3][4][5][6][7][8]

    Among the more surprising findings from the tournament were:

    the degree to which simple training exercises improved the accuracy of probabilistic judgments as measured by Brier scores;[3][4]
    the degree to which the best forecasters could learn to distinguish many degrees of uncertainty along the zero to 1.0 probability scale (many more distinctions than the traditional 7-point verbal scale used by the National Intelligence Council);[4][9]
    the consistency of the performance of the elite forecasters (superforecasters) across time and categories of questions;[4][5][6]
    the power of a log-odds extremizing aggregation algorithm to out-perform competitors;[7][10] and
    the apparent ability of GJP to generate probability estimates that were "reportedly 30% better than intelligence officers with access to actual classified information."
    Tetlock and Mellers[11] see forecasting tournaments as a possible mechanism for helping intelligence agencies escape from blame-game (or accountability) ping-pong in which agencies find themselves whipsawed between clashing critiques that they were either too slow to issue warnings (false negatives such as 9/11) and too fast to issue warnings (false positives). They argue that tournaments are ways of signaling that an organization is committed to playing a pure accuracy game –and generating probability estimates that are as accurate as possible (and not tilting estimates to avoid the most recent “mistake”).

    They have a website which allow individuals and groups to engage in prediction forecasting on a huge variety of topics across the world - economic, defense and political in nature.

    Kaufmann states that humans tend to focus on singular big ideas and develop coherent stories based on them which gives them confidence on future forecasting (just look at how experts performed). Also, we develop coherent stories on past events with hindsight which creates the illusion that past events were really predictable all along and this feeds into the delusion of future event forecasting. The reality is that future prediction is dependent on the certainty of events that are usually highly unpredictable (that seems absurdly simple) but everyone still slips into the delusion that they can outperform a monkey with darts once discussion gets going on future events.


  • #2
    Forgot this
    Tetlock conducted a set of small scale forecasting tournaments between 1984 and 2003. The forecasters were 284 experts from a variety of fields, including government officials, professors, journalists, and others, with many opinions, from Marxists to free-marketeers.

    The tournaments solicited roughly 28,000 predictions about the future and found the forecasters were often only slightly more accurate than chance, and usually worse than basic extrapolation algorithms, especially on longer–range forecasts three to five years out. Forecasters with the biggest news media profiles were also especially bad. This work suggests that there is a perverse inverse relationship between fame and accuracy.


    • #3
      Somehow I find this kind of a relief.


      • #4
        Most of the time its hard enough just to figure out what happened let alone predict anything.

        An understanding of the past gives us better tools to understand the future. To then go make predictions on the basis of it, how long out ? short term is possible, longer term is harder.

        so what then to make of the Chinese long game way of thinking ?


        • #5
          I am so glad these same "experts" are also involved in saving the planet from "climate change."
          "Only Nixon can go to China." -- Old Vulcan proverb.


          • #6
            Originally posted by Double Edge View Post
            Most of the time its hard enough just to figure out what happened let alone predict anything.

            An understanding of the past gives us better tools to understand the future. To then go make predictions on the basis of it, how long out ? short term is possible, longer term is harder.

            so what then to make of the Chinese long game way of thinking ?
            The Chinese long-game is a myth. Look at financial reform, for just one example: Lots of discussion over many years, but when it comes time to make a move, little happens.
            Trust me?
            I'm an economist!


            • #7
              Originally posted by Double Edge View Post

              how long out ? short term is possible, longer term is harder.
              Tetlocks original work focused on 3 to 5 year time ranges and he drew negative conclusions on expert ability to forecast. Later work focused on shorter ranges, inside twelve months.


              • #8
                Niall's batting for China these days

                China’s Understanding of Global Order Shouldn’t Be Ours | FP | Jan 24 2018

                Contrary to what Niall Ferguson suggests, if the liberal international system gets reduced to economics, it will no longer exist.
                BY AARON FRIEDBERG | JANUARY 24, 2018, 9:00 AM

                Niall Ferguson is a prolific public intellectual who has made a career out of shattering shibboleths. At various points he has defended the achievements of the British Empire, argued that the United Kingdom’s entry into the First World War was “the biggest error in modern history,” and made the case at length that Henry Kissinger is a misunderstood idealist.

                Ferguson is at it again. In a recent op-ed, he takes on what he describes as the prevailing “myth of the liberal international order.” This piece appeared in China’s Global Times, a pugnacious nationalistic tabloid published by the official People’s Daily. It is not surprising, then, that the view it presents happens to conform quite closely with the official Chinese Communist Party line. It is also, in important respects, mistaken and misleading.

                According to Ferguson, the notion that a liberal international order was created at the end of World War II by far-sighted British and American statesmen determined to avoid “the terrible mistakes of the 1930s and 1940s” is “a fairy tale” and “a historical fantasy.” Rather than “a truly international order,” what emerged after the war was a “bipolar order” that divided the world into “two empires.”

                He argues that it was only in 1991, with the collapse of the Soviet Union and the end of the Cold War, that it became possible to build a genuine liberal international order, one characterized by “truly free trade, truly free capital flows and large-scale migration across borders.” This new order was “hugely beneficial” to China, but one of its primary pillars was also what Ferguson describes as a “harmonious relationship” between China and America.

                In Ferguson’s telling, the collapse of that new order began in 2008 with the onset of the global financial crisis. The lingering aftereffects of the crisis fueled “popular disillusionment with globalization” and cleared the way for the eventual election of U.S. President Donald Trump. In response to what Ferguson describes as the recent American “volte-face” on trade, Beijing has sought to advance its own “alternative vision of international order,” most notably in Chinese President Xi Jinping’s 2017 speech at Davos in which he “made the case for globalization.”

                Looking ahead, Ferguson sees only two scenarios. In one, disputes over trade and geopolitics cause the United States and China to fall into a tightening “Thucydides Trap” that could well end in a catastrophic conflict. In the other, the two Pacific giants “recognize their common interests” and find ways to work together.

                The first thing to note about this account is how it echoes Beijing’s own narrative. Both portray the United States as a self-interested imperial power, albeit a rather incompetent one. Today, it is America’s alleged turn toward protectionism that threatens to destroy the open global trading system. By contrast, China’s wise leaders helped save the world from an even worse disaster in 2008 by unleashing a massive stimulus program, and they are now trying to save it again by championing the cause of continued economic openness. Like Ferguson, Chinese pundits and propagandists are eager to emphasize the supposed implications of the “Thucydides Trap”: The United States must abandon its confrontational ways and seek closer cooperation with China. The only alternative is war.

                There are a number of obvious problems with all this. To begin at the beginning: Ferguson uses the term “liberal international order” to refer to a global system characterized by unrestricted flows of goods, capital, and people. But an international order does not have to be global in scope, and a liberal order, properly understood, involves far more than just free trade.

                Despite the fond hopes of some Western statesmen (and, as Ferguson rightly observes, the retrospective imaginings of many contemporary political scientists), what emerged from the ashes of the Second World War was not a full-fledged global system built on liberal principles. But neither was it an empire like the one established by Moscow on its side of the Iron Curtain.

                What took shape in the West in the postwar era was a partial, geographically limited liberal international order: a system of states that would eventually comprise North America, most of Western Europe, and much of East Asia. While it initially contained a few illiberal outliers, the members of this system were increasingly drawn together by a shared commitment to certain universal principles (including the importance of individual freedom, political equality, democratic self-government, and the rule of law), by alliances and multilateral consultative mechanisms, and, in time, by widening flows of trade and investment.

                At the end of the Cold War, the United States and its allies decided to waive what had become, in effect, a requirement for full membership in the Western system in order to incorporate both Russia and China as fully as possible within it, despite the fact that neither had yet made a successful transition to liberal democracy. The assumption was that integration would speed that transition.

                It is the unintended consequences of this strategy, and ultimately its failure, that lie at the heart of the current crisis.

                Engagement with the West has enabled China to grow richer, more quickly than would otherwise have been possible. Instead of loosening its grip, however, the Communist Party regime has become even more repressive and militantly nationalistic, tightening control over the internet, launching a program that will use big data to track the political reliability of every Chinese citizen, and jailing yet more dissidents, lawyers, and alleged “separatists” in Xinjiang and Tibet.

                Rather than move toward an increased reliance on markets, the Chinese government has doubled down on its use of mercantilist policy tools, subsidizing “national champions” and enabling the acquisition of technology and intellectual property by all means, fair or foul, while continuing to regulate and restrict foreign imports and investments. What Xi has in mind when he sings the praises of “globalization” is not a level playing field but a situation in which China is able to persist in these practices while preserving the greatest possible access to the economies and societies of its open, liberal trading partners. Increasing frustration with this state of affairs has contributed to growing dissatisfaction, not only in the United States, but also in the other advanced industrial democracies.

                Ferguson overstates the extent to which the American people are turning away from free trade; polls show support has actually increased since Trump’s election. But he is surely right that the 2008 financial crisis was one of the factors contributing to the growth of populist sentiment and the questioning of liberal values and institutions across the West. What he fails to mention is that 2008 also marked the beginning of a shift toward a far more ambitious and at times aggressive Chinese foreign-policy posture, a trend that has accelerated since Xi’s rise to power in 2013. In the wake of the global financial crisis, Chinese strategists concluded that the United States had entered into a period of accelerating relative decline and that the time had come for them to act more assertively. This new spirit was on display in 2010, when China’s foreign minister warned his Southeast Asian counterparts to fall in line because “China is a big country and other countries are small countries, and that’s just a fact.”

                In recent years, China has deployed its growing military capabilities in an effort to exert control over virtually all of the waters and resources off its eastern coasts. As suggested by its disruption of trade and tourism with South Korea following Seoul’s 2017 decision to permit deployment of elements of a U.S. missile defense system, Beijing has become more open in using a mix of economic threats and inducements to try to drive wedges between Washington and its allies — and more transparent about its desire to displace the United States as the preponderant regional power.

                Even as it continues to exploit Western-designed institutions meant to promote economic openness (like the World Trade Organization) and to weaken those (like the United Nations Commission on Human Rights) that might criticize its domestic practices, Beijing is building new multilateral mechanisms (like the Asian Infrastructure Investment Bank) that will enhance its own power and influence at the expense of the United States and its liberal democratic allies. On orders from Xi, the Communist Party has intensified the use of propaganda, bribery, and disinformation aimed at weakening Western resistance to its ambitions. In the past decade, China has also moved into closer alignment with Russia, which is conducting its own campaign of intimidation and penetration against the democracies.

                While cooperation is always worth seeking, the divergence in interests and values between the democracies and the authoritarian powers will likely continue to limit its scope. This does not mean that war is inevitable. To the contrary, the best way for the Western powers to reduce the risk of conflict is to look to their own defenses, strengthening their ability to resist military coercion, political subversion, and economic exploitation. The liberal international system that the United States and its allies built in the wake of World War II is not global and may never be. But it is not a myth; it is a worthy accomplishment, and it deserves to be defended.

                Aaron L. Friedberg is Professor of Politics and International Affairs at Princeton University, where he has taught since 1987, and co-director of the Woodrow Wilson School’s Center for International Security Studies.
                Niall's op-ed in the GT


                • #9
                  ADB Outlook, 2020

                  The Asian Development Bank expects Real GDP in East Asia (China, Korea, Taiwan, Mongolia, Hong Kong, Macau) to grow just 2% in 2020, down from 5.4% in 2019. Fiji (-4.9%), Thailand (-4.8%), and Palau (-4.5%), will contract the most among the 46 economies that make up the ADB’s Asia and the Pacific region. Other significant drops include Hong Kong (-3.3%) and Samoa (-3%).

                  Fast growth is expected in Bangladesh (7.8%), Turkmenistan (6%), and Tajikistan (5.5%). Only Bhutan is expected to grow faster in 2020 than in 2019 (5.2%, up from 4.4%).

                  China is expected to grow just 2.3% in 2020, the slowest rate in the Reform Era. Exports are expected to decline by 8.5%, and imports by 7.8%.

                  The cut-off date for the analysis was March 19, 2020.
                  Trust me?
                  I'm an economist!