My solution was to merge the cells by their positions, then created a cell (reference to the first block of the merged cells) to assign a value and then set the alignment throught the CellUtil
// Merges the cells
CellRangeAddress cellRangeAddress = new CellRangeAddress(start, start, j, j + 1);
sheet.addMergedRegion(cellRangeAddress);
// Creates the cell
Cell cell = CellUtil.createCell(row, j, entry.getKey());
// Sets the allignment to the created cell
CellUtil.setAlignment(cell, workbook, CellStyle.ALIGN_CENTER);