For those who don’t know, Pat Metheny is one of the preeminent jazz guitarists of his, or any other, generation. He is the rare jazz musician who is able to sell out auditoriums. Case in point: On his live recording, The Road to You, you can hear the crowd actually sing the melody to one of his more Odyssean pieces, “Minuano.” His music is an amalgamation of influences, and he has recorded in a number of different genres: samba, progressive rock, and free jazz (refer to ’90s Song X recorded with Ornette Coleman) to name a few. Likewise, he performs in different settings: sometimes in a combo, either as a front or as a sideman; sometimes with the Pat Metheny Group, his extremely successful band; and sometimes on solo recordings, which up until now have focused on exploring the sonic possibilities given just one guitar (even a 42-string, custom-made one).

But with Orchestrion, Metheny has gone further than any previous solo recording. “It all kind of started with this fascination I had with a player piano my grandfather had in his house when I was little,” he said when his tour made its Chicago stop May 10. Taking the player piano idea and running with it, Metheny, along with what must be a team of ridiculously skilled engineers, has produced what he calls an “Orchestrion,” a two-story high structure like a wall of instruments: piano, guitar, electric bass, cymbals and drums of all kinds, mallet percussion, and bottles tuned to blow different pitches—all of which is controlled by the guitarist himself through the use of pedals and solenoid technology. It is a wild departure for even this musician who’s no Luddite when it comes to the use of music technology.

But is the end result effective? Live in concert, it surely is. After a lead-up of three acoustic pieces, Metheny unveiled his Orchestrion to great effect, using loops and pedals to create songs from scratch (I hesitate to say “organically,” given machines played all the instruments). But in exchange for the visceral wonder afforded the eye by the blinking lights and whirs of Metheny’s contraption, the music of Orchestrion comes nowhere close to Metheny’s best compositions. Like 2005’s The Way Up, one of the best long-form jazz pieces I’ve heard and the Pat Metheny Group’s most ambitious recording, the music of Orchestrion is composed as a suite. But where The Way Up alternately soared and sank to moody depths, Orchestrion often sounds like a MIDI recording over which a singular talent solos.

Of course, one must understand the parameters with which we must judge Orchestrion. As an alternative to the Pat Metheny Group, it of course fails spectacularly—programmed drums cannot at all compete with the talents of Antonio Sánchez on drums, and Lyle Mays’s rich coloring is a foundation on which PMG’s music is built. But I don’t think we’re being called upon to judge between PMG and Pat Metheny and his Orchestrion: They are, of course, different settings.

In judging this particular music, one has to consider the intricacy with which the music was made, the enormous talent and know-how that comes with being able to manipulate a piece of technology so adeptly so as to create improvised pieces using multiple instruments—yes, the instrument’s digitally controlled, but it’s difficult for the artist nonetheless. And, as Metheny has written, “whether it is developed and performed acoustically or otherwise…good notes are good notes.” Even if there are good notes in Orchestrion, though (and there are), the main draw is the Orchestrion itself. The live concert experience in this case is notably superior to the record.

That being said, does this new technology have any place in a jazz concert? After all, as puerile as it sounds, isn’t jazz about sharing in some sense, in creating a cohesive whole out of disparate parts—the talents and musical inclinations of separate individuals? As Metheny played that Monday night, he would look to each instrument as if to cue it. On occasion he would nod, as if he enjoyed the sound that was being produced. I kept asking myself why, and then came a possible answer. In the absence of someone else playing on stage with him, he had nowhere else to turn. It is impossible, after all, to look directly at oneself.