The question is stated as delivery of a "S-300 or S-400 missile system." A "system" requires all the major operational components to be delivered (search radar, targeting radar, missile launcher, command post, etc.). Regarding hiding the system, while I agree that generally you don't want to give away the exact location of your military assets, it is important to credibly signal your enemies if you expect to deter them. Most of the Iranian comments to this point are probably inward directed, to create a sense of strength in the aftermath of the nuclear deal with the West. Outward directed statements (to Israel) will come when the system is actually functioning.
@Clairvoyance The crux of my complaints against GJopen is that the research underlying it by Dr. Tetllock et al has not, to my knowledge, judging by their publications and news stories, ever addressed the problem of how to present questions to the crowd so as to enable the crowd to make the best possible forecasts. By contrast, their competitors in the ACE tournament, Scicast Predict, have published their research into this issue. They are competing for a CREATE contract, and teaming with Cultivate Labs, to my knowledge. So hang on, we may soon have a far better playground.
The resolution source is by far the most critical portion of a question and must be rigorously crafted to ensure there is no ambiguity when resolving it. Each user who reads a question will have his or her own interpretation of it and what could constitute a resolution; it is the job of question management to minimize these instances…. Adherence to a consistent style guide is necessary to ensure high-quality questions. Concise grammar, diction, and unambiguous explanations are absolutely necessary [pg. 88, 89, SciCast Annual Report, (2015) 25-May-2015, Year 4]
I, too, am glad to hear that the S-300 question will be voided. I assume my Brier score will be better after the score on it is backed out, but more importantly my trust in the process and morale will be better.
Even more importantly, I take this question as providing many examples of deliberate dissemination of misinformation--something that I knew went on but I had not been able to identify before.
@clairvoyance: I think that leaving a question open is not sufficient when there has been a significant event. I certainly agree that more communication would be beneficial between administrators and forecasters. Because silence implies that the status quo is still operative, I think that occasionally it would be appropriate for administrators to announce something to the effect: "We are aware of the reports linked and are evaluating their relevance for the resolution of this question." To me, this says, "No decision has been made yet regarding the significance of these reports. Do not assume that we have decided that we consider this event to NOT have resolved this question, AND do not assume that we will decide that this event HAS resolved this question. Please continue to forecast on this question and provide documentation to support your forecast(s)."
Exactly. As usual, they should just do what I told them to do weeks ago, instead of doing it now. Seriously. Just put me in charge and the world will be a better place.
This question should resolve as NO. As of December 31, 2015 there were no verifiable, reputable, open source confirmations that either S-300 or S-400 missile systems were delivered to Iran. The sale dating back to 2007 has been fraught with delays between the two parties and due to other world powers' interpretation of sanctions and defensive weaponry. Combined with an obsolete model, the need to retrofit, an 8 billion dollar lawsuit by Iran for breach of contract, etc., despite claims by parties on both sides of the principals who stood to gain by portrayal of steady progress in delivery, the facts remain that the work order for the factory, the specifications for the upgrade to reflect new technologies and the removal of the lawsuit are all active issues, and as of the period of this question, unresolved. The question therefore resolves to No. << what is so hard about that?
Ravel: it would be nice for our brier scores, but since GJOpen closed the question before December 30th, all they can do now is void the question. Basically it wouldn't be fair to resolve it as "No," because people couldn't update their forecast between the time that GJO closed the question and the end of 2015.
This is why I would have preferred that they left the question open to forecasting until the expiration date. If they had, then we could have gotten credit for our "No" forecast. If need be, we could have waited until January or February for a definitive answer, and then calculated our brier scores retroactively.
@Clairvoyance, exactly my sentiments on early closing. If Iran disappears into a giant sinkhole, then several questions could be voided. Short of that, let it ride until closing date.
Yeah, my thoughts exactly. With highly controversial questions: let the question ride to the expiration date, and then we can deal with the resolution.
Just some thinking out loud. It would nice to have a master list of all of the sick and twisted errr dedicated folks here. One that you copy to a notepad or a clipboard for easy reference.
Nota bene/catching up - Heffalump, @000, @Phronesis and I were all in the same prediction market. We had 303 in our instance per the post mortem report. @cmeinel and @morrell were in the same individual Brier instance. Their format had 1800+ in terms of the post mortem report. Our format dropped to around 50 or so active participants by March. Heff may have better data on this. One simple conclusion is that the individual Brier instances had higher drop out rates than prediction markets.
Reaffirming my forecast.
The question is stated as delivery of a "S-300 or S-400 missile system." A "system" requires all the major operational components to be delivered (search radar, targeting radar, missile launcher, command post, etc.). Regarding hiding the system, while I agree that generally you don't want to give away the exact location of your military assets, it is important to credibly signal your enemies if you expect to deter them. Most of the Iranian comments to this point are probably inward directed, to create a sense of strength in the aftermath of the nuclear deal with the West. Outward directed statements (to Israel) will come when the system is actually functioning.
@Clairvoyance The crux of my complaints against GJopen is that the research underlying it by Dr. Tetllock et al has not, to my knowledge, judging by their publications and news stories, ever addressed the problem of how to present questions to the crowd so as to enable the crowd to make the best possible forecasts. By contrast, their competitors in the ACE tournament, Scicast Predict, have published their research into this issue. They are competing for a CREATE contract, and teaming with Cultivate Labs, to my knowledge. So hang on, we may soon have a far better playground.
Specifically, from pg. 88, 89, SciCast Annual Report, (2015) 25-May-2015, Year 4, http://blog.scicast.org/2015/09/04/scicast-final-report-released/
The resolution source is by far the most critical portion of a question and must be rigorously crafted to ensure there is no ambiguity when resolving it. Each user who reads a question will have his or her own interpretation of it and what could constitute a resolution; it is the job of question management to minimize these instances…. Adherence to a consistent style guide is necessary to ensure high-quality questions. Concise grammar, diction, and unambiguous explanations are absolutely necessary [pg. 88, 89, SciCast Annual Report, (2015) 25-May-2015, Year 4]
I apologize for the infantile humor. I could not resist. @cmeinel's infamous "third hand:"
https://www.youtube.com/watch?v=wW8LvUvTZhA
@redacted - you can be serious and have fun too. I think it helps memory retention.
I, too, am glad to hear that the S-300 question will be voided. I assume my Brier score will be better after the score on it is backed out, but more importantly my trust in the process and morale will be better.
Even more importantly, I take this question as providing many examples of deliberate dissemination of misinformation--something that I knew went on but I had not been able to identify before.
@clairvoyance: I think that leaving a question open is not sufficient when there has been a significant event. I certainly agree that more communication would be beneficial between administrators and forecasters. Because silence implies that the status quo is still operative, I think that occasionally it would be appropriate for administrators to announce something to the effect: "We are aware of the reports linked and are evaluating their relevance for the resolution of this question." To me, this says, "No decision has been made yet regarding the significance of these reports. Do not assume that we have decided that we consider this event to NOT have resolved this question, AND do not assume that we will decide that this event HAS resolved this question. Please continue to forecast on this question and provide documentation to support your forecast(s)."
Exactly. As usual, they should just do what I told them to do weeks ago, instead of doing it now. Seriously. Just put me in charge and the world will be a better place.
I NEED A HEAD SHRINKER IN AISLE 4!
This question should resolve as NO. As of December 31, 2015 there were no verifiable, reputable, open source confirmations that either S-300 or S-400 missile systems were delivered to Iran. The sale dating back to 2007 has been fraught with delays between the two parties and due to other world powers' interpretation of sanctions and defensive weaponry. Combined with an obsolete model, the need to retrofit, an 8 billion dollar lawsuit by Iran for breach of contract, etc., despite claims by parties on both sides of the principals who stood to gain by portrayal of steady progress in delivery, the facts remain that the work order for the factory, the specifications for the upgrade to reflect new technologies and the removal of the lawsuit are all active issues, and as of the period of this question, unresolved. The question therefore resolves to No. << what is so hard about that?
Ravel: it would be nice for our brier scores, but since GJOpen closed the question before December 30th, all they can do now is void the question. Basically it wouldn't be fair to resolve it as "No," because people couldn't update their forecast between the time that GJO closed the question and the end of 2015.
This is why I would have preferred that they left the question open to forecasting until the expiration date. If they had, then we could have gotten credit for our "No" forecast. If need be, we could have waited until January or February for a definitive answer, and then calculated our brier scores retroactively.
@Clairvoyance, exactly my sentiments on early closing. If Iran disappears into a giant sinkhole, then several questions could be voided. Short of that, let it ride until closing date.
Yeah, my thoughts exactly. With highly controversial questions: let the question ride to the expiration date, and then we can deal with the resolution.
Just some thinking out loud. It would nice to have a master list of all of the sick and twisted errr dedicated folks here. One that you copy to a notepad or a clipboard for easy reference.
@Heffalump - Amish!? You sure don't act Amish. https://www.youtube.com/watch?v=OZZSfGdXIf8
Nota bene/catching up - Heffalump, @000, @Phronesis and I were all in the same prediction market. We had 303 in our instance per the post mortem report. @cmeinel and @morrell were in the same individual Brier instance. Their format had 1800+ in terms of the post mortem report. Our format dropped to around 50 or so active participants by March. Heff may have better data on this. One simple conclusion is that the individual Brier instances had higher drop out rates than prediction markets.