Miles Davis was only two years old in 1928, when James Weldon Johnson wrote his famous essay, “The Dilemma of the Negro Author,” outlining the problem of the African-American artist who commands the loyalty of both black and white audiences but harbors very different feelings toward each. “The double audience,” as Johnson called it, was Davis’s dilemma throughout his long career.

That Davis distrusted whites is clear from the countless cuts left by his razor tongue – wounds so numerous and (sometimes) unfair that critics still call him a racist. He wasn’t a racist, unless by over-use the word is stripped of all meaning. He was a proud prickly character, or, if you prefer, an arrogant cuss, who (to quote one of his favorite expressions) “didn’t take no shit off nobody.” He was also a consummate artist who strove to overcome his own biases and fathom why whites as well as blacks loved his music.

Davis’s career, which crossed many genres and market niches, began in July 1944, when the legendary Billy Eckstine band visited St. Louis. Booked into a white club, Eckstine was fired for using the front door instead of the “colored” rear entrance. So he took his amazing band, which included Charlie Parker, John Birks “Dizzy” Gillespie, and Art Blakey, to a Negro establishment, the Riviera Club, where they played with fiery beauty before a large enthusiastic black audience – and hurled a lightning bolt of inspiration at an eighteen-year-old trumpeter named Miles Davis.

Years later Davis would confess, “I’ve come close to matching the feeling of that night in 1944 in music, when I first heard Diz and Bird, but I’ve never quite got there.” A strange statement, coming from a figure so full of musical refulgence. What kind of lightning did Davis mean? And why did it strike only once?

The answer lies in Davis’s lifelong struggle to achieve three goals: high musical art, broad popular success, and a deep connection with his fellow African Americans. Given the times he lived through, it is hardly surprising that he rarely achieved all three simultaneously.

In 1946, when Davis went to New York to apprentice himself to Parker and Gillespie, bebop was still popular in Harlem. But moving downtown it attracted whites, some true believers but others thrill seekers of the type Ralph Ellison had in mind when he called Parker “a sacrificial figure whose struggles against personal chaos, onstage and off, served as entertainment for a ravenous, sensation-starved, culturally disoriented public.” It was toward such ignorant white fans that Davis expressed his coldest, back-turning contempt.

As bebop grew more virtuosic, many black listeners sought the simpler pleasures of rhythm and blues (R & B), leaving a core of loyal fans but no reliable market niche. Add the ravages of heroin, Parker’s death, and the condescension of the mainstream, and one can see why Davis’s career almost ended in early 1950s.

But then he landed on his feet, with a new sound that made him a celebrity. Critics still debate the cool style pioneered by Davis in collaboration with Gil Evans. Max Harrison praises its “muted colors” and “lucid proportions”; Stanley Crouch dismisses it as “another failed attempt to marry jazz to European devices.” Still, all parties agree that cool was popular with whites – especially Kind of Blue (1960), whose mass appeal would today place it right next to the chai tea in Starbuck’s.

As cool grew “whiter” in the hands of California musicians Dave Brubeck, Paul Desmond, and Chet Baker, the dense percussive style known as hard bop became the “black” alternative. Yet this racial divide did not affect Davis, because as Gary Giddins notes, “The warring subcultures, West Coast jazz (cool) and East Coast jazz (hard bop) had the same midwestern parent: one Miles Dewey Davis.” To the yin of cool Davis brought rich sonority, blues feeling, and swing; to the yang of hard bop he brought stillness, melodic beauty, and understatement. By refusing to color-code either his music or his audience, Davis rose at the age of thirty-four to the summit of artistic excellence.

Then the ground shifted. The 1960s saw three dramatic changes: first, the transformation of jazz into an esoteric art music; second, the emergence of a large racially mixed audience for popular music; and third, a new mode of music-making that (inadvertently) helped to subordinate the aural to the visual, the ear to the eye. For Davis these changes were further complicated by the problem of the double audience.

Consider first the transformation of jazz. Long before the 1960s, jazz musicians had exploited modernist ideas like chromatic harmony, modal scales, and electronics. In the 1960s the New Thing led by Ornette Coleman, John Coltrane, and Cecil Taylor went further, expanding the sound vocabulary of instruments, eliminating cadential harmony and the modal system, exploring polytonality and atonality, adopting irregular meter, and finally abolishing metric time. The goal, turbo-boosted by black political activism, was total improvisatory freedom.

Some jazz elders were dismayed. Growled Coleman Hawkins in 1964, “Those guys are looking for a gimmick, a short cut. There is no short cut.” And Davis himself quipped that if Taylor was “what the critics are digging,” then “them critics better stop having coffee.” But here again, Davis landed on his feet. With four fiercely gifted young players – Ron Carter, Herbie Hancock, Wayne Shorter, and Tony Williams – he set off a controlled explosion that equaled the excitement of the New Thing but through a disciplined route that brooked no short cuts.

It would have been the perfect solution, had Davis’s audience-seeking antennae not told him that jazz was being eclipsed by popular music. Never one to assume that “the fewer hear you, the better you are,” he went where the action was – to people like Jimi Hendrix and Sly Stone, who wielded shamanistic power over the racially mixed mass audience. In 1969 it was too late to ride the jubilant crest of the counterculture, so Davis made Bitches Brew, whose bottomless, shifting ostinatos, eddying scraps of guitar, snarling bass clarinet, and dissonant chords are the perfect backdrop for his pitchfork-thrust horn.

Bitches Brew still conjures the turbulent undertow of the late 1960s, but it is not rock. When rock musicians tried to evoke the same dark mood, they did so by goosing the volume, setting off smoke bombs, and applying too much eye makeup. Bands like Black Sabbath were an instant hit with the junior testosterone crowd, and in the 1970s their monstrous offspring, heavy metal, became one of the most durable genres in the history of popular music. But this has nothing to do with Davis.

More relevant is the third change. Inspired by Indian raga and other Eastern sources, composers like Philip Glass, La Monte Young, Terry Riley, Steve Reich, and Mike Oldfield began in the 1960s to fashion a sound, all too familiar nowadays, in which a clear steady pulse is blended with repetitive, often tape-looped melodic-harmonic fragments. The aim, in Reich’s words, was to “facilitate closely detailed listening." But the result was nearly the opposite. Beginning with the use of Oldfield's Tubular Bells (1973) on the soundtrack of The Exorcist, minimalist-derived music became “aural wallpaper” for an increasingly image-driven culture.

This sheds new light on Davis’s “jazz-rock fusion.” Mid-1970s albums like Dark Magus, Agharta, and Pangaea take little from jazz, apart from free improvisation (which Davis had resisted a decade earlier), and little from rock, apart from ear-bleeding volume and electronic instruments. What these experiments really reveal is what Amiri Baraka calls Davis’s “penchant for minimalism.”

Davis’s minimalism deepened over time. He always preferred sketched understatement to embellished overstatement. But in his pre-fusion music, the repetition of simple melodic motives is always related to the underlying harmonic structure; and even within the loose boundaries of modal improvisation, his melodic line always suggests a fuller volume, as does a late drawing by Matisse. It is only in his post-fusion phase that Davis’s melodies quit evoking larger forms and become recycled fragments.

What does all this have to do with the double audience? By an odd convergence, both the minimalists and the black nationalists of the 1960s sought to liberate music from shapely melody, which they variously defined as exhausted, oppressive, and Eurocentric. To be fair, Davis bought into this trend less by rejecting melody than by embracing rhythm. Echoing the militant mood, he claimed rhythm as his racial birthright and griped that “white people were trying to suppress rhythm because of where it comes from – Africa.”

Davis knew this was an oversimplification. “We ain’t in Africa,” he said in a different context, “and we don’t play just chants. There’s some theory under what we do.” The African-American music that is his true birthright is, like the infant brought before King Solomon, a living whole in which even the zestiest dance number partakes of melody, and even the tenderest ballad partakes of rhythm. By 1975 this living whole was being sundered in the name of “liberation,” and to the extent that Davis assisted the sacrifice, he sold out his rarest gift. It is probably no accident that he quit music that year.

By the 1980s, when Davis made his comeback, almost all popular music had been “liberated” from any melody more taxing than what record producers call “hooks.” It was not Davis’s fault that the art of composing, playing, or listening to melodies longer than two bars seemed as dead as the fugue. The damage was done; all Davis could do was recognize it. “A lot of people ask me where music is going today,” he said. “I think it’s going in short phrases. If you listen, anybody with an ear can hear that.”

Davis came back with the double audience firmly in mind. Playing to the mass audience, he granted celebrity interviews and made music videos. Playing to black youth, he mixed it up with contemporary R & B and hip-hop. But what about his art? Are critics right to say that he squandered his melodic gift, chasing “short phrases” down the musical drain?

The answer is yes – at first. Listening to Davis’s horn pick its lonely way through a state-of-the-art sonic landscape on such R & B albums as Tutu (1986) and Amandla (1989) is like watching a pro tennis player take the court against a ball-serving machine. No matter how skillfully the pro returns the serve, there’s no excitement, no volley – because one of the players isn’t a player.

Why did Davis submit to the high-tech yoke, when by his own admission he preferred “live, raunchy, get down” music? For the same reason he chose, back in the 1970s, to dress as a superannuated super-fly and put a blaxploitation cartoon on the cover of On the Corner (1972): he yearned to connect with black youth. That effort backfired, probably because many young African Americans shared the perception of the writer David Nicholson: “Fusion Miles always seemed like one of my uncles in bell bottoms.”

By 1990 Davis was chasing a different youth trend: hip-hop. Inspired by Quincy Jones’s Back on the Block, he undertook a collaboration with rapper Easy Mo Bee that resulted in Doo-Bop, an album released after Davis’s death that impresses neither as hip-hop nor as jazz. Still, Doo-Bop did not pander to the gangsta image then grabbing the headlines. Crouch has accused post-fusion Davis of being a “licker of moneyed boots.” But he didn’t lick Dr. Dre’s Nikes.

Nor was that the end of the story. In the mid-1980s Davis formed a new band with alto saxophonist Kenny Garrett, bass guitarist Foley, keyboardist Adam Holzman, and drummer Ricky Wellman. This galvanizing, globe-trotting band knit the raveled strands of fusion into tight, powerful pieces whose full impact was not felt until 1996, when Holzman and Gordon Meltzer assembled eleven of the best unedited tracks for Live Around the World, a remarkable album that has caused many nay-sayers to look again at Davis’s later career.

Did Miles Davis finally retrieve the lightning, that original thunderbolt of high art, broad acceptance, and black solidarity that struck him long ago in 1944? Maybe not, but for his tireless struggle to do so, we can only admire him. With extraordinary courage and agility he eluded most (not all) of the dangers strewing his path, the many shapes of figurative death that stalked his musical generation. His long struggle to stay on top was messy and unseemly at times, and it may be that by his refusal to age gracefully he contributed to the decline of American music. But that decline would have happened anyway, and I for one lack the presumption to blame a man for not walking in a straight line across a minefield.

This article appeared originally in the New York Times