Input based data fetch optimization

Question

Is there a cleaner/ better readable way to maintain and iterate many-many mapping rather than multiple if conditions ? (One way would be to store the mapping of ReportType -> DataSet Req. in the database)

For e.g if there's an application which depending upon 'reports selected', generates reports using data from required database tables. Something like this:

Now since the data to be fetched in these tables is huge and shared for multiple reports, we came with a not so good looking approach to optimize getData function using if() conditions like this :

getDataForRequiredReports(List requiredReports) {
 Map<String,Object> dataMap;

 if(requiredReports.contains(ReportB))
    dataMap.add(newEntry("marketData", DAOlayer.get("MarketData")));

 if(requiredReports.contains(ReportA) || requiredReports.contains(ReportB) || requiredReports.contains(ReportE))
    dataMap.add(newEntry("pastData", DAOlayer.get("PastData")));

 if(requiredReports.contains(ReportC) || requiredReports.contains(ReportD) || requiredReports.contains(ReportE))
    dataMap.add(newEntry("manualData", DAOlayer.get("ManualData")));

 if(requiredReports.contains(ReportC) || requiredReports.contains(ReportE))
    dataMap.add(newEntry("mlOutputData", DAOlayer.get("MLOutputData")));

 return dataMap;
}

Possible duplicate of [Style for control flow with validation checks](https://softwareengineering.stackexchange.com/questions/148849/style-for-control-flow-with-validation-checks) — gnat, Jun 25 '18 at 12:34
I recon you're doing the work in the wrong place. Do the data build in the DW based upon each report required. You're not going to win pulling large amounts of data up to a memory-constrained layer if the data truly is that huge. Pragmatism over purity here. A bit old-school I know... — LoztInSpace, Jul 04 '18 at 10:25
@LoztInSpace I consent on that but I'm doing this since the DW is managed by separate team and due to auth limitations we are not allowed to run queries directly. — pxm, Jul 05 '18 at 15:36
Why `ReportC` doesn't need `pastData` ? (so far I understand it should because of inheriting `ReportE`) — Spotted, Jul 10 '18 at 12:19

score 2 · Answer 1 · answered Jul 04 '18 at 08:59

Like this, you're attempting to write conditions based on what data you'll need, when you should be writing it in terms of the data required for each individual report.

Consider the following:

Map<Class<? extends Report>, ReportHandler> reportHandlers = new HashMap<Class<? extends Report>, ReportHandler>();

Map<String,Object> getDataForRequiredReports(List<Report> requiredReports) {
    Map<String,Object> dataMap = new HashMap<String, Object>();

    for(Report report : requiredReports) {

        ReportHandler handler = reportHandlers.get(report.getClass());

        for(String entryName : handler.getEntryNames()) {
            if(!dataMap.containsKey(entryName)) {
                dataMap.put(entryName, handler.newEntry(entryName));
            }
        }
    }
    return dataMap;
}

You evaluate each report to get the necessary components for that particular class, adding only if not already present.

Here's an example of a possible ReportAHandler:

public class ReportAHandler implements ReportHandler {
    private static final Map<String, String> NAME_QUERY_MAPPING = new HashMap<String, String>();

    static {
        NAME_QUERY_MAPPING.put("pastData", "PastData");
    }

    public Set<String> getEntryNames() {
        return NAME_QUERY_MAPPING.keySet();
    }

    public Entry newEntry(String entryName) {
        return newEntry(entryName, DAOlayer.get(NAME_QUERY_MAPPING.get(entryName)));
    }
}

In this way you load the information exactly once for each necessary data of each report. The only thing which changes from report to report is the contents of NAME_QUERY_MAPPING, so you could probably generalize behavior for each report, unless you have reason to add a more complicated logic later.

score 1 · Answer 2 · answered Jul 10 '18 at 10:50

When mapping professional information you should try to make it as clear and readable as possible.

DAO-Types should be a central entity:

public enum StoredDataType {
    MARKET_DATA  ("marketData",   "MarketData"),
    PAST_DATA    ("pastData",     "PastData"),
    MANUAL_DATA  ("manualData",   "ManualData"),
    MLOUTPUT_DATA("mlOutputData", "MLOutputData");

    public Reports(String entry, String dao)
}

Reports should clearly define required data:

public enum Report {
    REPORT_A(PAST_DATA),
    REPORT_B(MARKET_DATA, PAST_DATA),
    REPORT_C(MANUAL_DATA, MLOUTPUT_DATA),
    REPORT_D(MANUAL_DATA),
    REPORT_E(MANUAL_DATA, MLOUTPUT_DATA, PAST_DATA);

    public Report(StoredDataTaype... requiredData)
}

And your logic can easily create the required Entries:

Arrays.stream(StoredDataType.values())
    .filter(d -> requiredReports.stream().anyMatch(r -> r.requires(d)))
    .forEach(d -> dataMap.add(newEntry(d.name, d.daoName)));

Input based data fetch optimization

2 Answers2