# Split data

Students need to split the data by implementing the `split()` method. This takes as input the `data` that needs to be split, together with its corresponding `labels`. The split is based on three parameters:

> - `apps_train` specifies the apps used for both training and testing. The given `ratio` of flows for these apps should be in `data_train` and `1-ratio` of flows for these apps should be in `data_test`.

> - `apps_test` specifies the apps used only for testing, all flows of these apps should be in the resulting `data_test` and no flows flows of these apps should be in the resulting `data_train`.

> - `ratio` specifies the ratio of flows from `apps_train` that should be in `data_train`. The remaining flows should be in `data_test`.

Note

Students may expect that an app `A` is specified in either the `apps_train` or `apps_test` but not both. I.e., the union of `apps_train` and `apps_test` is the complete set of apps and the intersection of `apps_train` and `apps_test` is empty.

`Assignment.``split`(data, labels, apps_train, apps_test, ratio)

  Extract features from each flow. Must be implemented by the students.

  Parameters

    - data (np.array of shape=(n_flows, n_features)) – Data for which to produce a split.

    - labels (np.array of shape=(n_flows,)) – Labels corresponding to the data.

    - apps_train (list()) – List of apps to use for training.

    - apps_test (list()) – List of apps to use for testing.

    - ratio (float) – Ratio of data for training apps to use for training. The rest should be used for testing.

  Returns

    - X_train (array-like of shape=(n_flows_train, n_features)) – Datapoints selected for training.

    - y_train (array-like of shape=(n_flows_train,)) – Corresponding labels selected for training.

    - X_test (array-like of shape=(n_flows_test, n_features)) – Datapoints selected for testing.

    - y_test (array-like of shape=(n_flows_test,)) – Corresponding labels selected for testing.

Students are not required to return a numpy array. To ensure that everything still works when students return a list we use the `scale_numpy` function that transforms their output of `scale` to numpy arrays.

`Assignment.``split_numpy`(data, labels, apps_train, apps_test, ratio)

  Perform split function and transform output to numpy arrays.

  Parameters

    - data (np.array of shape=(n_flows, n_features)) – Data for which to produce a split.

    - labels (np.array of shape=(n_flows,)) – Labels corresponding to the data.

    - apps_train (list()) – List of apps to use for training.

    - apps_test (list()) – List of apps to use for testing.

    - ratio (float) – Ratio of data for training apps to use for training. The rest should be used for testing.

  Returns

    - X_train (np.array of shape=(n_flows_train, n_features)) – Datapoints selected for training.

    - y_train (np.array of shape=(n_flows_train,)) – Corresponding labels selected for training.

    - X_test (np.array of shape=(n_flows_test, n_features)) – Datapoints selected for testing.

    - y_test (np.array of shape=(n_flows_test,)) – Corresponding labels selected for testing.

We automatically apply the split on the data from `benign.csv` for a given `apps_train` (which automatically computes the `apps_test`) and a given `ratio`.

`Assignment.``get_split`(apps_train, ratio)

  Returns the split of ‘benign.csv’ file for a given input.

  Parameters

    - apps_train (list()) – Apps to use in the training set, all apps not in apps_train will be used only for the testing set.

    - ratio (float) – Ratio of datapoints for apps in app_train to use for training.

  Returns

    - X_train (array-like of shape=(n_samples_train, n_features)) – Datapoints used for training.

    - y_train (array-like of shape=(n_samples_train,)) – Labels corresponding to training data. +1 if the corresponding app is in the training dataset, -1 if the corresponding app is not in the training dataset, I.e., all values are +1.

    - X_test (array-like of shape=(n_samples_test, n_features)) – Datapoints used for testing.

    - y_test (array-like of shape=(n_samples_test,)) – Labels corresponding to testing data. +1 if the corresponding app is in the training dataset, -1 if the corresponding app is not in the training dataset.

## Test

We provide some test cases for students to check their implementation of `split()`. The `test_split` method runs each test case and `assert_split` checks whether each test case is correct.

`Assignment.``test_split`(verbose=False)

  Executes several test cases for splitting the data.

  Parameters

    verbose (boolean, default=False) – If True, print tests that are executed.

`Assignment.``assert_split`(original_data, original_labels, data_train, labels_train, data_test, labels_test, expected_samples_train, apps_test)

  Tests whether the split is correct. If an incorrect split is found, it raises an error with a suggestion for the students to debug their code.

  Parameters

    - original_data (np.array of shape=(n_samples, n_features)) – Original data before split.

    - original_labels (np.array of shape=(n_samples, n_features)) – Original labels before split.

    - data_train (np.array of shape=(n_samples_train, n_features)) – Data selected for training.

    - labels_train (np.array of shape=(n_samples_train,)) – Corresponding labels selected for training.

    - data_test (np.array of shape=(n_samples_test, n_features)) – Data selected for testing.

    - labels_test (np.array of shape=(n_samples_test,)) – Corresponding labels selected for testing.

    - expected_samples_train (int) – Number of samples expected to be in training.

    - apps_test (list) – List of apps that should only appear in testing dataset.