Edit 20.4.2021 – Two years later, two years wiser
As this question/answer has gotten lots of attention and is still valid after all the years I wanted to throw few pointers. Most of the details underneath are is still valid. Still, I would direct towards using VueX with Lodash (or modern version of native JS functions) when dealing with filtered results & complex objects.
In order to ease the stress of your backend you can keep things simple: Fetch plain objects without related models. This means that your main results have only ID-keys to related objects. Use Axios or similar library to fetch all the related data with separate AJAX-requests (“customers”, “projects”, “locations”) and use VueX to store ’em in their own list-properties. Create getters for each, such as:
projectsById: state => {
return _.keyBy(state.projects, "id")
},
This way you can use related models for fetching labels and/or full objects when required and your backend doesn’t need to fetch related data more than once. States and getters will be available within micro-components as well.
Basically: Avoid fetching full model-trees (even though C# EF or PHP Laravel provide tools for ’em) when dealing with large datasets. Use atomic approach: Fetch 20 different lists (“Axios.all([…])” is your friend!), each with own controller endpoint and cache the results to VueX store… And have fun 😉
Edit 12.03.2019 – additional tips at the end of this answer
It’s been a while since I asked this question and I finally got to optimize this part of my project. I’d like to give few pointers for anyone having these performance and/or memory-issues.
Vue documentation never really explained it, but as Andrey pointed out you CAN use the component-object as an data-storage for your custom objects & object-lists. After all, it’s just an normal javascript-object.
After optimization my list component setup looks somewhat like this:
module.exports = {
items: [],
mixins: [sharedUtils],
data: function() {
return {
columns: {
all: []
etc... Lot's of data & methods
The items-array is filled with thousands of complex objects (about 80mb of data, 6mb compressed) which I’m handling as non-reactive. This proved to be less of an issue than I would have thought — Instead of using v-for directly against items I was already using structure in which I triggered filtering of this array whenever user clicked some filter-button and/or inputted string-filtering (such as name). Basically this “processFilters”-method goes through non-responsive items-array and returns filteredItems, which is stored in data-context. Thus it automatically becomes reactive as it’s mutated.
<tr v-for="item in filteredItems"
This way all the items within filteredItems stay reactive, but also lose reactivity when they are filtered out, thus saving bunch-load of memory. Whopping 1200mb shrunk to 400mb, which was exactly what I was looking for. Clever!
There are few issues which need to be addressed. Since items doesn’t exist in data-context you cannot use it directly within template. This means that instead of writing…
<div v-if="items.length > 0 && everythingElseIsReady">
… I had to store length of items-array to separate data prop. This could have been fixed with computed value as well, but I like to keep those properties existing.
Giving up the reactivity of your main data-array isn’t such a bad thing after all – The most important part is to understand that modifications which are made directly against items within that base-array are never triggering any changes to UI and/or sub-components (douh). This shouldn’t be such an issue as long as you separate your code in such a way that you have “hidden data container” which holds all the results from backend, and you have smaller (filtered) presentation array of that large container. By using good REST-architecture you should already be good to go with non-reactive data-storage, as long as you remember to check that after saving the item within non-reactive data storage has also been updated to latest revision.
Additionally I was baffled by how little it matters performance-wise how many micro-components there are against hundreds of rows. The render takes a hit obviously, but even if I were to pass large props thousands of times (as I have thousands of instances of input-cells) it didn’t seem to hit the memory. One of this kind of objects is my global translations-key/value-pair object, having over 20 000 lines of translated strings… but it still didn’t matter. This makes sense, as Javascript uses object-references and Vue Core seems to be properly coded, so as long as you use such configuration objects as props you are simply referring from thousands of objects to the same data-set.
Finally, I’d say start going crazy with complex CRUD objects without fear of hitting memory limit!
Huge thanks for Andrey Popov for giving nudge towards right direction!
Tips (12.03.2019)
Since it’s been a while and as I have continued building UI’s with large & complex datasets I decided to drop few short ideas & tips.
- Consider how you manage your master-records (ie. persons or products) vs related records (sub-objects / relational objects). Try to limit the amount of data injected for subcomponents, as you might be representing the same sub-object multiple times for different master-records. The problem is that it’s possible that these objects are not actually reference-objects!
Consider situation where you have person-object, which contains city-object. Multiple persons live in the same city, but when you fetch JSON-data from backend are you sure are those duplicated city-objects actually one and same city (shared/referenced city-object between persons), or multiple representations of similar object (with data being exactly same, but under the hood each one being an individual instance / unique object). Let’s say that you have 50 000 persons, each one containing the same sub-object/property “city”: { id: 4, name: “Megatown” }, did you just fetch 50 000 individual city instances instead of just one? Is person1.city === person2.city , or do they just look the same and still be two different objects?
If you are unsure whether you are refering to shared city-object or using dozens of instances of similar sub-objects you could simply do there referencing inside your person-list-component. Your person contains city-id, so fetch list of cities with separate REST-method (getCities), and do the pairing on UI-level. This way you have only one list of cities, and you could resolve city from that that list and inject it to person, thus making reference to only one city. Alternatively you could resolve the city from list and pass it as an property to your person-component.
Also make sure to consider what is the purpose of the sub-object. Do you need it to be reactive, or is it static? In order to save bunch of memory you could just tell “person.city = city”, which will be injected for each and every person-component, but if it needs to be reactive then you need to use Vue.set -method… and remember that if each city needs to be own instance (so that each person has similar city-object, but properties need to be editable per person) then you need to make sure that you are not using referred object! Thus you most likely need to clone the city-object, which will eat up browsers memory.
- Your micro-component might contain separate view-states for both read-only-state and editor-state. This is quite common. Still, you are actually creating instance of that micro-component every time when, thus initializing that component thousands of times.
Think of situation where you have Excel-like spreadsheet with table and table-rows. Each cell contains your custom “my-input” -component, which takes “readonly”-property from your layout. If the UI is on the readonly-state then you are displaying only the label part inside that my-input-component, but otherwise you are displaying input-tag with some special conditions (such as having different input for datetime, number, text, textarea, select-tag etc). Now let’s assume you have 100 rows with 20 columns, so you are actually initializing 2000 my-input-components. Now the question is — what could be improved (performance-wise)?
Well, you could separate readonly-label from my-input-component to your list-view, so that you either display readonly-version (label) OR you you display the editable my-input-component. This way you have v-if condition, which makes sure that those 2000 micro-components won’t be initialized unless you have specifically requested to initialize ’em (due either row or whole layout moving from readonly -> editable -state)… You probably guess how big the impact is memory-wise for browser, when Vue doesn’t need to create 2000 components.
If you are facing that your page loads really slow it might not be VUE at all. Check out the amount of HTML-tags rendered to your HTML. HTML performs rather poorly when you have large amounts of tags. One of the simplest ways to demonstrate this is by repeating select-tag with 2000 options 100 times, or by having a single 20000 option select-tag. The same way you might be overflowing the amount of html-tags by having lots of micro-components with unnecessary wrapping divs etc… The less depth and less tags you have, the less rendering performance is required from browser & CPU.
Try to learn good HTML-tag architecture via examples. For an example you could study how Trello -services dashboard-view has been programmed. It’s quite simple and beautiful representation of rather semi-complex service, with minimal amount of sub-divs.
There are many ways to improve memory handling, but I’d say that most important ones relate to separating “hidden” objects from visible objects, as described on my original answer. Second part is understanding the difference or instanced vs referenced objects. Third is to limit the amount of unnecessary data-passing between objects.
Personally I haven’t tried this, but there exists a Vue-virtual-scroller component which handles any amount of data by simply being a wrapper for seemingly infinite amounts of data. Check out the concept @ https://github.com/Akryum/vue-virtual-scroller , and let me know if it solved the problem for you.
I hope these guidelines give some ideas for optimizing your components. Never give up the hope, there is always room for improvement!