how to add values to an array of objects dynamically in javascript?

You have to instantiate the object first. The simplest way is:

var lab =["1","2","3"];
var val = [42,55,51,22];
var data = [];
for(var i=0; i<4; i++)  {
    data.push({label: lab[i], value: val[i]});
}

Or an other, less concise way, but closer to your original code:

for(var i=0; i<4; i++)  {
   data[i] = {};              // creates a new object
   data[i].label = lab[i];
   data[i].value = val[i];    
}

array() will not create a new array (unless you defined that function). Either Array() or new Array() or just [].

I recommend to read the MDN JavaScript Guide.