Why Array.prototype.reduce() is not taking an empty array as accumulator?

When you try to do return filteredArr.push(collectionElement), in essence you are returning length of filteredArr after the push operation. The push() method adds one or more elements to the end of an array and returns the new length of the array.
Ref: Array.prototype.push().

You need to return the filteredArr from your anonymous function, so that it is used as the previousValue for the next call

var collection = [3, 5, 11, 23, 1];

// filter all the elements bigger than 10 to a new array

var output = collection.reduce(function(filteredArr, collectionElement) {
  if (collectionElement > 10) {
    filteredArr.push(collectionElement);
  }
  return filteredArr;
}, []);