1966 stories
·
2 followers

From Naija to Abidjan

1 Share

One country is Anglophone, and the other is Francophone. Still, there are between 1 to 4 million people of Nigerian descent living in Côte d'Ivoire today.


alt
Adjamé district, Abidjan. Image credit Cyprien Hauser via Flickr CC BY-ND 2.0.

Lire en français ici.

There’s a saying—if there’s a place in this world where you don’t find Nigerians, it’s not a place suitable to live. Côte d’Ivoire is no exception. There are between 1 and 4 million people of Nigerian descent living in Côte d’Ivoire today, including Yorubas, many of them from Ejigbo in Osun State, as well as Igbo and Hausa ethnic groups.

The first wave of Nigerian resettlements began more than a century ago—in 1902, before either country became an independent, autonomous state. A couple of decades later, many Igbos continued to settle in the francophone nation after the Biafran War—at the time, former Ivorian president Félix Houphouët-Boigny had an agreement with Chukwuemeka Odumegwu Ojukwu, the first president of Biafra, to let him and his people settle in Côte d’Ivoire. The Igbo community still lives there to this day, among other Nigerians. Some of them were born in Côte d’Ivoire, and others migrated there, either alone or with their families. In Treichville, Abidjan, there’s even a neighborhood formerly known as Biafra. In Abidjan, Nigerians have their own hot spots, such as Bomakouté Ayass, a market in Adjamé, or the Yassonde maquis (a type of Ivorian bar) in Adjamé.

However, despite having a reputation for being very welcoming, Côte d’Ivoire, like all countries, is not immune to xenophobia. This tension goes both ways: the 2023 AFCON revealed both solidarity as well as divisions between countries, and before, during, and after the finale, some Nigerians expressed dismay and ignorance toward Côte d’Ivoire, a fact made even more baffling as millions of Nigerians live in the country. Ignorance on social media is nothing new, but it was still sad to see, as relations between Nigeria and Côte d’Ivoire precede colonialism, and there have been many cultural exchanges between the countries.

Nigerians in Côte d’Ivoire mainly live in Abidjan, in neighborhoods like Adjamé or Yopougon. Olawalé Ibrahim, a businessman, is one of them. “I came to visit Côte d’Ivoire and then stayed. One day I’ll go back to Nigeria to settle down. I still have my house in Nigeria,” he explained. Ibrahim arrived in Abidjan in 1994. “Many other Nigerians in the country have a different opinion from mine. They have spent more time in Côte d’Ivoire than Nigeria and would rather stay there. They can’t go back to Nigeria; they’d prefer to die here. Côte d’Ivoire is our second country.”

Ibrahim recalls that when former Ivorian president Laurent Gbagbo came to power in 2000, many foreigners felt targeted by his policies. It reached a heightened point during the 2002 civil war. At that time, the Ivorian national radio would often broadcast xenophobic messages. His supporters went as far as looting shops owned by Burkinabé people. “Most Ivorians are pretty open-minded people, but politicians have impacted the way people think,” Ibrahim added.

Economic duress is one of the reasons why so many Nigerians are moving to Côte d’Ivoire. Ayanjimi-Jai, a used-clothes fashion seller, has lived in Côte d’Ivoire since 1993. “In Lagos, it was hard to plan my life, as there were many struggles to get fuel, electricity, and water. There were more facilities in Côte d’Ivoire,” he explained. “I used to be in the navy, and then I came to Côte d’Ivoire and never left.” Like many migrants, he faced difficulties, among them learning the language. “French is a very difficult language for me as an English-speaking person,” he says. Despite the language barrier, Jai chose Côte d’Ivoire as a country to live in over the neighboring anglophone country of Ghana. In his view, Ghana’s issues with public services such as electricity and water were too similar to those of his homeland of Nigeria.

Professor Bell Ihua, a Nigerian pollster, pan-Africanist, and the executive director of Africa Polling Institute (API), has written about Nigerian communities in Côte D’Ivoire. He explains another surprising reason behind the increase in Nigerians in Côte d’Ivoire: “Church expansion is another reason behind these migration waves,” Ihua says. “There are branches of those big churches from Nigeria that are now being exported in Côte d’Ivoire, such as the Redeem Church and the Christ Embassy.”

These waves of migration in Côte D’Ivoire have also brought an Ivorian influence to Nigeria. Professor Ihua mentions Ejigbo, a Nigerian town in Osun state with strong cultural ties with Côte d’Ivoire, where French is the second most-spoken language. Most of the people there have traveled to Côte d’Ivoire at some point or have family members there, naturally integrating French into their social fabric over the years. There are also some Ivorians living in Ejigbo. “You’ll find elders born in Côte d’Ivoire who are French speakers because of that long connection, even naming their children French names, eating Ivorian food such as attiéké and sauce gombo,” Professor Ihua explains, adding that the local architecture is influenced by Côte D’Ivoire. “They even have their own version of the allocodrome [open-air Ivorian takeaway shops], where people come to buy their attieké and plantain.” According to Professor Ihua, there are up to five buses linking Ejigbo to Abidjan every week, further indicating that the community feels connected to both Ivorian and Nigerian culture.

Ihua believes that the Nigerian migration to Côte D’Ivoire has increased over the years. “Igbos have continued to travel as well as Yorubas from Ejigbo,” he says.“Migration in Nigeria has been compounded by economic and social issues. Young Nigerians who might not be able to settle in Western countries might consider traveling to other West African countries instead.”

The relationship between both countries seems to be growing. “Diplomatically speaking, Nigerians do not forget the support given to Biafra during the civil war, and [the relationship] will continue to grow as a positive one,” Ilhua says. Cultural exchange goes both ways: Ayanjimi is proud to have taught Ivorians more about Nigerian culture. At present, Ilhua has no intention of returning to Nigeria and believes he will probably stay in Côte d’Ivoire. “Côte D’Ivoire is my home, and I am Ivorian in spirit,” he says.

African nations were created based on arbitrary borders and divisions depending on which European countries colonized them—and the borders created by the Berlin Conference of 1885 started the process of European leaders safeguarding their interests, dividing Africa’s land into “artificial nations,” as Tanzanian President Julius Nyerere once said. One of the unfortunate consequences of these arbitrary borders is that so many cultural and human exchanges between countries are prevented due to language and cultural barriers—it is substantially easier for francophone Africans to exchange with each other than it might be for an anglophone African and a francophone one. Despite this, Africans in the continent move between different countries and increase their understanding of the other. Nigerian communities have fostered connections in Côte d’Ivoire, where some of them have lived for decades, and set up lives abroad for, one might hope, the better. These multigenerational bonds might help set a template for future cultural exchange throughout the continent, unencumbered by the limitations of colonial structures.


Read the whole story
koranteng
3 days ago
reply
Share this story
Delete

Buyer's Remorse (Part 3 The Situation Thereof)

1 Share
Southern stress tests... Part 3 of Buyer's Remorse

III. The Situation Thereof


Suffice to say that the claims adjuster had been rather impressed
With the way the tribes of Agona had confronted the heart of darkness
Far from succumbing to the new variants introduced by their tormentor
Precious few of their settlements had turned into new epicenters

The lack of casualties had left observers perplexed
For they could no longer invoke their savior complex
The routine lectures and platitudes they picked off the shelf
Would now face the rejoinder: physician, heal thyself

It was one thing to acknowledge their dire state of poverty
But watching the rest of the world succumb to excess mortality
Had left them open to the possibilities of renewed confidence
Especially as they realized the wages of observed competence

True there were those who emphasized the youth of their populace
Discounting the pragmatism with which they had countered the menace
Pointing to their lack of facility with the ins and outs of accounting
Damning them with faint praise, their success was said to be confounding

They'd accepted leftovers from Tedros of Who and kept a low profile
No, he gave them all due credit, they'd paid penance contra denial
But the relative respite begged the question: were their rulers legitimate?
Their sole remaining assets, Asase Yaa's excessive liability certificates

As he read the scrolls, he perused the situation reports
Making sure to cross check what he'd found in the courts
Still, the stress test of humanity he'd initiated was dragging on
He wondered if the dwellers of the torrid zone could withstand the situation

But perhaps all the claims adjuster needed to do was to bear witness
The cast of characters in their lands were known for their brazenness
The chief of Rata, glad to be back in power, had promptly reverted to embezzling
Despite all the repeated warnings about the rising threat of inflation

To the bill of goods that was the routine economic mismanagement
One could always find those prone to succumb to martial entanglements
The proud scribes of Piatra agitated for war and bit off more they could chew
Their warriors were roundly defeated in pitched battles and had to be rescued

But most troubling was the situation in Santara which had come to a boil
To prevent a return of hunger season, their Okyeame banned the export of palm oil
His faint hope was that this structural adjustment was worthy of Machiavelli
But with these leading indicators on hand the claims adjuster now had a casus belli

What, after all, was the proposed basis of this new normalcy
To turn back the clock to a time when life was carefree?
Or to build a new way of life cognizant of the new restrictions?
A dilemma that had all the makings of a chicken and egg situation

Would they learn their lessons and heed the mosquito principle
Or give in to insatiable appetites as if they were invincible?
The gods would be watching closely to see if they reached a consensus
For it was an article of faith even to the gods to protect the least of us

There was enough of an opening in these reports of dysfunction
For the claims adjuster to reassert to them that no one was coming
And to those who would suggest that the situation thereof was unfair
He had the ready retort: a barking dog does not lie in the hyena's lair


Haste not in life



Situation, a playlist


A soundtrack for all types of situations. The claims adjuster is on the case. (spotify version)

Buyer's Remorse (Series Index)


A covidious folktale
  1. Buying Opportunity
  2. Tempting the Town Crier
  3. The Situation Thereof
  4. Fair Warning

This folktale is part of a series: In a covidious time.

Next in Part 4: Fair Warning

File under: , , , , , , , , , , , , , , , , , ,

Writing log. May 5, 2022

Read the whole story
koranteng
5 days ago
reply
Share this story
Delete

Cracking an old ZIP file to help open source the ANC's "Operation Vula" secret crypto code

1 Share

It's not often that you find yourself staring at code that few people have ever seen, code that was an important part in bringing down the apartheid system in South Africa, and code that was used for secure communication using one-time pads smuggled into South Africa by a flight attendant on floppy disks. But I found myself doing that one morning recently after having helped decrypt a 30 year old PKZIP file whose password had long been forgotten.

Some time ago I became interested in the secure communications used by the ANC as part of Operation Vula in the late 1980s. Operation Vula was the infiltration of ANC leaders (and materiel) into South Africa to set up an underground network bringing together the various elements of ANC activism operating inside the country. 

The operation needed secure communications to be successful and these were established using 8-bit computers, DTMF tones, acoustic couplers and a variety of other equipment for exchanging one-time pad encrypted messages using programs written in PowerBASIC.

I won't go into the detail of how this worked as Tim Jenkin, the person primarily responsible for the encryption system, has now open sourced the original code, and which can be found here. Tim's write up on the encryption system can be found here. I thoroughly recommend reading it for the details.

The code hadn't been open sourced before for one simple reason: on leaving the UK for South Africa in 1991 he had zipped up all the source code and set a password on it. In the intervening years he'd simply forgotten the password! So, when I emailed him to ask if it had been open sourced he replied:

I still have the Vula source code but unfortunately most of it I can't access because when I left the UK in 1991 to return to South Africa, I zipped up all the files with a password. I was able to decode and extract one of the files but it was a very early version of the software. The rest I couldn't extract because I forgot the password. When I got back to SA there was no need to access the code. I thought I would never forget the password but when I tried to decode it a few years later, I couldn't remember it.

If you could find out how to decode zipped files that would release the software, which I would be more than happy to share. I have made a few attempts to crack the code but so far have been unsuccessful.

I readily agreed and he sent me two files: ALLBAS.ZIP and CODMAY93.ZIP. These were both created with an early version of PKZIP and had passwords set on them. Luckily, there is a known plain text attack against the ZipCrypto scheme used in that era's ZIP format. And an open source implementation of the attack called bkcrack.

So, it was a "simple matter" of predicting 12 bytes of plain text at a known location inside the ZIP file. Here's a sample of what's inside the ZIP file:

$ bkcrack -L ALLBAS.ZIP | head -n 20

bkcrack 1.7.0 - 2024-05-26
Archive: ALLBAS.ZIP
Index Encryption Compression CRC32    Uncompressed  Packed size Name
----- ---------- ----------- -------- ------------ ------------ ----------------
    0 ZipCrypto  Shrink      b0f86b1d          163          117 A1PSW.BAS
    1 ZipCrypto  Shrink      8fa662d4          163          118 A2PSW.BAS
    2 ZipCrypto  Shrink      0c5a7295          163          119 A3PSW.BAS
    3 ZipCrypto  Shrink      49907f86          179          125 A4PSW.BAS
    4 ZipCrypto  Shrink      3d20eb7a          163          120 A5PSW.BAS
    5 ZipCrypto  Shrink      f8b558f0          136          128 BIOS.INC
    6 ZipCrypto  Implode     799074ed          377          278 CHKERR.INC
    7 ZipCrypto  Implode     c44ea0a5        17906         5401 CODSUBS.INC
    8 ZipCrypto  Implode     7bd7e23d        27287         8297 COMAID.BAS
    9 ZipCrypto  Implode     03dc63da         2109         1001 COMKEY.BAS
   10 ZipCrypto  Store       3500d320         2372         2384 CONFIG.TIM
   11 ZipCrypto  Shrink      35a85089          147          111 CONPSW.BAS
   12 ZipCrypto  Implode     55be75ce         2094          825 DOS.INC
   13 ZipCrypto  Shrink      3387d043          134          127 DOSVER.INC
   14 ZipCrypto  Implode     28a32efa         1304          535 DOSX.INC
   15 ZipCrypto  Implode     6578a66c         3196          966 EDDY.BAS

Tim had some unencrypted .BAS files lying around but they were different versions than those in the file and trying the bkcrack attack using them (after running them through original PKZIP in DOSBox) was unsuccessful and I decided to apply some brain power before attempting further attacks.

ALLBAS.ZIP contained a number of files that were uncompressed, because they were already binary and not worth compressing. These files are marked as Store:

$ bkcrack -L ALLBAS.ZIP | grep Store
   10 ZipCrypto  Store       3500d320         2372         2384 CONFIG.TIM
   23 ZipCrypto  Store       14a285ac            2           14 KEYCOD.EXE
   25 ZipCrypto  Store       d6343ce1         4767         4779 KEYONE.ZIP
   26 ZipCrypto  Store       650778b7         6523         6535 KEYTHREE.ZIP
   30 ZipCrypto  Store       12a711cd        58172        58184 OLDCOD.ZIP
   41 ZipCrypto  Store       00000000            0           12 TAPCOD.EXE
   44 ZipCrypto  Store       55000714        12716        12728 TECOD5.ZIP
   45 ZipCrypto  Store       f4f4366c         9230         9242 TECOD6.ZIP

Files that are Store'd are fruitful for plaintext prediction because they have not undergone compression and there's no need to have the original file to compress in order to obtain plaintext. Focussing on the ZIP files, since the ZIP files start with a PK header, seemed like a good place to find predictable plaintext at a known position. Here are the fields in the standard PK header at the very start of a ZIP file:


A viable attack appeared to be to predict the name of the first file in the archive. If the file name was at least 8 characters (which seemed pretty easy since at least four characters were used for .BAS, .INC etc.) then at least 12 characters of plaintext would be available when the file name size (offset 0x1A, 0x1B) and the length of the extra field (which appeared to be 0x00, 0x00 in all the ZIPs Tim sent) was added.

In the worst case, it would be possible to bruteforce the potential names of files given that they all appear to be uppercase/number combinations with a maximum length of eight characters plus extension. But that turned out not to be necessary.

Happily, Tim had a different version of OLDCOD.ZIP (one of the ZIP files inside ALLBAS.ZIP) and was able to tell me that the first file in it was COMKEY.BAS. So, I whipped up a quick Perl program to create the necessary plaintext in that hope that the OLDCOD.ZIP inside ALLBAS.ZIP did start with COMKEY.BAS:

$ cat maken.pl
use strict;
use warnings;

my $outfile = "hexname-$$.txt";

while (<>) {
    chomp;
    my $bas = $_;
    print("$bas / $outfile\n");
    my $n = sprintf("%c\x00\x00\x00$bas",length($bas));
    open G, ">$outfile";
    print G $n;
    close G;
    system("bkcrack -C ALLBAS.ZIP -c OLDCOD.ZIP -p $outfile -o 26 -j 8");
}

23 minutes later bkcrack spit out the key to the ALLBAS.ZIP file and was able to decrypt it. The same key worked for CODMAY93.ZIP also.

$ time echo "COMKEY.BAS" | perl maken.pl

COMKEY.BAS / hexname-41227.txt

bkcrack 1.7.0 - 2024-05-26

[07:49:38] Z reduction using 6 bytes of known plaintext

100.0 % (6 / 6)

[07:49:38] Attack on 925073 Z values at index 33

Keys: 98e0f009 48a0b11a c70f8499

80.6 % (745571 / 925073) 

Found a solution. Stopping.

You may resume the attack with the option: --continue-attack 745571

[18:13:49] Keys

98e0f009 48a0b11a c70f8499


real 23m4.371s

user 162m3.520s

sys 0m37.752s

bkcrack does the decryption once the key has been found:

$ bkcrack -C ALLBAS.ZIP -k 98e0f009 48a0b11a c70f8499 -D ALLBAS-DECRYPTED.ZIP

bkcrack 1.7.0 - 2024-05-26

[07:52:22] Writing decrypted archive ALLBAS-DECRYPTED.ZIP

100.0 % (81 / 81)

$ bkcrack -C CODMAY93.ZIP -k 98e0f009 48a0b11a c70f8499 -D CODMAY93-DECRYPTED.ZIP

bkcrack 1.7.0 - 2024-05-26

[07:58:31] Writing decrypted archive CODMAY93-DECRYPTED.ZIP

100.0 % (40 / 40)


And with that the long encrypted source code used to help set up secure communications for the ANC was available!


Had that failed I was going to attack one of the other ZIP files using the same attack (before resorting to bruteforceing file names). I'd guessed that TECOD5.ZIP was probably a ZIP of just the file TECOD.BAS (or maybe TECOD5.BAS) based on the compressed size of TECOD.BAS in ALLBAS.ZIP). Turns out I wouldn't have had to wait 23 minutes if I'd started there:


$ time echo "TECOD5.BAS" | perl maken.pl

TECOD5.BAS / hexname-41544.txt

bkcrack 1.7.0 - 2024-05-26

[18:14:51] Z reduction using 6 bytes of known plaintext

100.0 % (6 / 6)

[18:14:51] Attack on 880113 Z values at index 33

Keys: 98e0f009 48a0b11a c70f8499

2.4 % (20737 / 880113)

Found a solution. Stopping.

You may resume the attack with the option: --continue-attack 20737

[18:15:29] Keys

98e0f009 48a0b11a c70f8499


real 0m38.152s

user 4m35.318s

sys 0m0.897s


The known plaintext attack against ZipCrypto works quickly with the right plaintext. If you ever have to do something similar it's worth spending time thinking about the plaintext. In particular, files that are Store'd in the ZIP file are useful to examine since they are uncompressed and it may be easier to predict their contents (rather than having to find an original file and compress it to match what's stored in the ZIP).


Running the code


I compiled two of the programs and ran them using DOSBox. The first, RANDOM.BAS, was used to create disks of random numbers to be used as a one time pad, the other, TECOD.BAS, was used to encrypt and decrypt messages sent via email. The code I compiled and the generated executables can be found here.


Compilation is simply running the PowerBASIC compiler as follows:


C:\>EXE\PBC TECOD.BAS PowerBASIC Compiler Version 3.00b Copyright (c) 1989-1993 by Robert S. Zale Spectra Publishing, Sunnyvale, CA, USA C:\TECOD.BAS 2575 statements, 2329 lines Compile time: 00:12.0 Compilation speed: 12600 stmts/minute 45984 bytes code, 4880 bytes data, 2048 bytes stack Segments(1): 46k C:\>EXE\PBC RANDOM.BAS PowerBASIC Compiler Version 3.00b Copyright (c) 1989-1993 by Robert S. Zale Spectra Publishing, Sunnyvale, CA, USA C:\RANDOM.BAS 2194 statements, 1940 lines Compile time: 00:10.1 Compilation speed: 12600 stmts/minute 33328 bytes code, 4704 bytes data, 3072 bytes stack Segments(1): 34k C:\>


The first step is to create random data on disk to be used as a one time pad. RANDOM.EXE uses three different randomness generating algorithms (one based on a random key you type in yourself).



Encryption and decryption is done through TECOD.EXE which is password protected. 


Although the password is embedded in the program and quite simple Tim Jenkin did obfuscate it as follows:

DIM PW$(PL)
PW$(9)=CHR$(66):PW$(4)=CHR$(66):PW$(1)=CHR$(84):PW$(5)=CHR$(79):PW$(2) = CHR$(73)
PW$(3)=CHR$(77):PW$(6)=CHR$(66):PW$(8)=CHR$(77):PW$(10)=CHR$(79):PW$(7)=CHR$(73)

In this particular version of the program that makes the password TIMBOBIMBO which when entered takes you to the main menu. Note that each version of these programs being sent to different members of the ANC had different passwords.

If you're interested in running these programs yourself, the manual is here.

Here are three short videos showing the creation of random data in RANDATA.1 for the key using RANDOM.EXE, followed by encrypting a message stored in PLAIN.TXT on a RAM disk (all crypto operations were meant to happen on a RAM disk) and turning it into PLAIN.BIN (and the reverse).

Creating random data to be used as an encryption key



Encrypting a file


Here the programs (TECOD.EXE/TECOD.CNF) are on a floppy disk in A:, the data disk (containing the key file created above) are in B: and there's a RAM disk on R:. To get this to work the RANDATA.1 file created in the step above needs to be renamed to SNUM. 



Decrypting a file


Here the programs (TECOD.EXE/TECOD.CNF) are on a floppy disk in A:, the data disk (containing the key file created above) are in B: and there's a RAM disk on R:. The RANDATA.1 needs to be called RNUM on B:. 


There are lots of interesting details of how these programs work that deserve another longer blog post when I have time. Or a detailed study by someone else. For example, the key material is destroyed after use, the RANDOM.EXE program has multiple ways of making randomness and code to check the distribution of the random bytes created. There's an emphasis on using the RAM disk for all cryptographic operations. 
Read the whole story
koranteng
6 days ago
reply
Share this story
Delete

Buyer's Remorse (Part 2 Tempting the Town Crier)

1 Share
A surge in the East... Part 2 of Buyer's Remorse

II. Tempting the Town Crier


This time around, the town crier beat the gong as he prepared to deliver the bad news
Quite simply the folks in Chwedru had reached the limits of zero tolerance
True, they had faithfully paid all premiums due for their soul insurance
But after two years of skimping on preparation, there was no valid excuse

The scribes were no fools, they realized they had to change course
For it would be no use at this stage to invoke the force majeure clause
It really didn't auger well that the gong was being beaten
That was surely an impending sign of a sticky situation

The common folk of the Wan tribe had long accepted the elders' authority
The unwritten contract was that human life, hard as it was, had sanctity
Most importantly, the hunger seasons of yore would never again be observed
If avoiding such troubles was no longer guaranteed, their power could not be preserved

Supply chains be damned, it wasn't even close to a dilemma
Their response was clear and swiftly conveyed to the town crier
They would immediately double down on the restrictions
And strongly recommend to all emotional vaccinations

Word was sent to the herbalists to produce new concoctions
All able bodied Wan should heed the call for battle stations
The standing army, long inactive, would be readied straightaway
And on the farms, reserve crops would be harvested without delay

The town crier, at length, conferred with the chief scribe and the linguists
These last had a weary air to them, you might call them pessimists
They had seen the rest of the world in upheaval, as if in a daze
And were proud they had so far managed to avoid even a second wave

Under the palaver tree, they debated the situation
Needless to say they had to take tough decisions
They tallied the upcoming festivals that would need to be postponed
Lest the entire tribe end up like those in the torrid zone

Once the course of action was decided there could be no second guessing
Their strategic resolve was unshakeable, there was no window dressing
The division of labor would fall into place, the rest was organization
Collective responsibility was the policy, they would brook no deviation

...

It was as he left the Wan settlements that the town crier was accosted
Ananse the Spider had decided that a direct approach was warranted
Without any preliminaries, he simply handed over a large monetary package
And asked straight up if that was sufficient compensation for the proposed damage

The town crier was rather impressed
   by the full frontal assault on his probity
He was sufficiently malleable by temperament
   (having what you might call moral flexibility)
He calmly assessed the assumed wages
   of this proposed integrity abasement
"Throw in a bottle of Schnapps and gourd of palm wine,
   that should be sufficient"
Good help came cheap these days, Ananse had to admit
There was almost no need to negotiate the misdeed
Just a few choice words as he explained the outline of his plan
And before long, he heard those three words: "I'm your man"

A goat doesn't pass a leopard's door, it wasn't mere survival bias
Such unerring instincts had long guided Ananse in his dealings
Limiting one's exposure, what some would term common sense
Surprisingly enough turned out to be uncommon in those lands

It's not so much that he was the greatest of grifters
The truth was that there would always be takers
Humanity was easily dazzled by rhetoric that sparkled
Hastening, as ever, to prove that gullibility was immortal


No problem



Temptation, a playlist


A soundtrack for this note. It's hard to resist the musical commentary. (spotify version)

Buyer's Remorse (Series Index)


A covidious folktale
  1. Buying Opportunity
  2. Tempting the Town Crier
  3. The Situation Thereof


This folktale is part of a series: In a covidious time.

Next in Part 3: The Situation Thereof

File under: , , , , , , , , , , , , , , , , , ,

Writing log. May 3, 2022

Read the whole story
koranteng
12 days ago
reply
Share this story
Delete

Polio in Gaza: A Jewish Fable

1 Share

You probably have to be Jewish to appreciate the full and bitter irony of this sentence:

Nearly 11 months into a devastating war, a serious new challenge has emerged in Gaza: polio.

Growing up, there were two uncontested heroes in the Jewish-American pantheon: Sandy Koufax and Jonas Salk. If you were really in the know, you’d add a third: Albert Sabin.

Salk invented the polio vaccine, Sabin invented the oral polio vaccine.

Now Israel has contributed an entirely new entry to the history of polio.

It almost reads like a fable from Jewish literature. Except it’s not.

Read the whole story
koranteng
15 days ago
reply
Share this story
Delete

horses for courses

1 Share

Apparently Nate Silver has a new book coming out, in which (if the reviews I’ve read are representative) he advances a theory of political philosophy based on his experiences in gambling.  It set me thinking, about the interesting negative template that’s always there when Data Guys get into gambling.  There’s always a lot of poker, maybe a little bit of backgammon – card counting at casinos maybe.  You have your Moneyball and its equivalents in sports betting.  But it surprises me that you very rarely see Data Science Skills applied to horse racing, even though that’s a very big and liquid gambling market where Big Data is quite easily available.

Subscribe now

Here's an exception that proves the rule – Bill Benter.  He used computer analysis to beat the bookies at the Hong Kong Jockey Club and took them for hundreds of millions of dollars (ie, peanuts to the Hong Kong Jockey Club).

And the reason that this is an exception which proves the rule rather than falsifying it is that this could only be done in Hong Kong.  Partly because of the way betting works there (it’s a peri-mutual system for those that care; nothing is really gained by knowing what that means other than that it makes it a bit easier to place large bets).  But mainly because Hong Kong is unique in having only two racetracks, one of which is much more important than the other.  It’s also too far away from other racing markets to regularly move horses there.  So it’s pretty much the only place in the word where the same horses race each other, on the same course, very frequently indeed.

This matters, because “horses for courses” isn’t just a fun rhyming thing to say – it’s absolutely fundamental to horse race gambling.  Some horses are good at shorter distances, some are stayers.  Some of them have the brains to adjust their pace on a hill, others will try to keep sprinting and exhaust themselves.  Some of them have a gait that suits muddy conditions, others run better on hard ground.  Et cetera … 

Racecourses are very different.  In the UK, for example, they aren’t even consistent as to whether the horses run clockwise or anticlockwise. The course at Chester is almost circular, which means that racing there is a test of how good a horse is on its front left foot.  Perth has a steep slope up to the finish which always catches out front-runners.  A sharp bend (like the one at Catterick) will confuse and unsettle a horse that hasn’t seen it before, but not one that’s expecting it.  Et cetera …

All of which means that horse racing data has an absolutely massive curse of dimensionality.  If you’re trying to estimate a model to use historical race data to tell you which horse out of a field of eight or nine is likely to show up fastest, then you’d have to, at the very least, take into account the course, distance, “going” (ground conditions) and class of each race.  Since there are also interaction effects (different courses will suit different horses depending on whether it’s muddy or dry, for example), then you’re eating up degrees of freedom very quickly. 

Added to which, there’s a curse of non-stationarity.  You want lots of data points to deal with your curse of dimensionality, but very few horses have more than a dozen starts in a season.  So the more data you collect, the older some of it is.  And a lot can change for a horse in a year – injuries, changes of trainer and the simple effect of maturity and aging.

This is why bookmakers smile when they meet a quant with a system, particularly if they hear that their new customer makes a lot of money playing poker.  Statistical analysis of the British racetrack is basically impossible…. Or is it?

The data issues I’ve outlined make it more or less impossible to get the kind of model that Moneyballers and poker players want, which gives you a single statistically optimal prediction.  But that doesn’t mean the formbook is useless.  There are all sorts of rules of thumb and statistical regularities that can be established.  (For example “The sharp turn at Catterick will always catch out a horse that hasn’t raced there before”, as I said a couple of paragraphs ago).

Really good racing analysts can systematically beat the odds by knowing a lot of these little rules of thumb, being really familiar with the form and having enough experience to know which statistical regularities are most salient for any given race.  They are also good at spotting which races are easiest to analyse and most likely to offer attractive odds, and good at not betting on the other ones.  (I’ll talk about my own system on Friday if you like!).

I think most of the world is more like horse racing than it is like poker. (I notice that when the Data Guys start trying to make money in finance, they tend to gravitate toward derivatives and high-frequency trading.  “Horses for courses” is a phrase you will often hear in the stock market, because companies and industries have even more variety than racecourses do).  That’s why most cybernetic schemes to improve the human condition tend to fail. 

And I think that’s my considered response to a lot of the comments on my James C Scott posts.  To a large extent, the debate about techne and metis, technical quantifiable knowledge versus embedded and tacit skill, is really about the curse of dimensionality.  There’s a point at which, to quote Ben Recht, “math becomes metaphor” and the skill of prediction and management is one of knowing which parts of the mental model to apply; while you learn these things in mathematical terms, the algorithm doesn’t help you any more.



Read the whole story
koranteng
17 days ago
reply
Share this story
Delete
Next Page of Stories