They produced a new record in 1996,with which they celebrate

with which = with the new record这里with表示“以...",表示celebrated的方式或伴随状语,其它介词没有这个功能啦
补充：整个是一个包含有限制性定语从句的主从复合句.把定语从句单独拉出来改写一下更好理they celebrated their former time as a real band with the new record . 但若这样写的话容易引起误解,所以就把with the new record 提到句首,来引导从句了.

从原句分析，这个句子的两个分句应该是：
they produced a new record in 1996. They celebrated the new recorde with their former time (or record?) as a real band. (与他们之前的时间（记录？）一起，他们像一只真正的乐队一样庆祝这一新纪录（的产生））。
抛开本句逻辑上是...