Decision Trees are robust and comprehensible machine learning models. They enable us to arrive at conclusions based on a sequence of if-else criteria. To retrieve decision rules from a Scikit-Learn Decision Tree, you can employ the export_text() method or navigate through the tree_ attribute programmatically.
In this article, we’ll discuss how to extract and interpret decision rules from a Scikit-Learn Decision Tree. We’ll also guide you through various Python codes to visualize and display the decision rules in a format understandable to humans. So, let’s dive in!
Extracting decision rules from decision trees is beneficial for:
Model Interpretability: Facilitates understanding of how a model formulates predictions.
Debugging: Assists in recognizing possible biases present in the training data.
Rule-based systems: Enables the utilization of decision rules for automated decision-making outside the constraints of machine learning models.
The following steps outline the process for extracting decision rules:
Step 1: Train a Decision Tree Classifier
In this initial phase, we will train a decision tree classifier using the Iris dataset.
Example:
Python
Code Copied!
var isMobile = window.innerWidth “);
editor89618.setValue(decodedContent); // Set the default text
editor89618.clearSelection();
editor89618.setOptions({
maxLines: Infinity
});
function decodeHTML89618(input) {
var doc = new DOMParser().parseFromString(input, “text/html”);
return doc.documentElement.textContent;
}
// Function to copy code to clipboard
function copyCodeToClipboard89618() {
const code = editor89618.getValue(); // Get code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert(“Code copied to clipboard!”);
// Function to run the entered code
function runCode89618() {
var code = editor89618.getSession().getValue();
jQuery(“#runBtn89618 i.run-code”).show();
jQuery(“.output-tab”).click();
function closeoutput89618() {
var code = editor89618.getSession().getValue();
jQuery(".maineditor89618 .code-editor-output").hide();
}
// Attach event listeners to the buttons
document.getElementById("copyBtn89618").addEventListener("click", copyCodeToClipboard89618);
document.getElementById("runBtn89618").addEventListener("click", runCode89618);
``````javascript
copyCodeToClipboard89618);
document.getElementById("runBtn89618").addEventListener("click", runCode89618);
document.getElementById("closeoutputBtn89618").addEventListener("click", closeoutput89618);
Result:
Analysis:
The preceding code is utilized to load the Iris dataset. It divides the dataset into training and testing subsets. Subsequently, it trains a Decision Tree Classifier with a maximum depth of 3. Finally, it displays the training and testing accuracy of the model.
Step 2: Visualizing the Decision Tree
Prior to extracting the decision rules, let’s visualize the tree structure:
Illustration:
Python
Code Copied!
var isMobile = window.innerWidth ");
editor21806.setValue(decodedContent); // Set the default text
editor21806.clearSelection();
editor21806.setOptions({
maxLines: Infinity
});
function decodeHTML21806(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to clone code to clipboard
function copyCodeToClipboard21806() {
const code = editor21806.getValue(); // Fetch code from the editor
navigator.clipboard.writeText(code).then(() => {
jQuery(".maineditor21806 .copymessage").show();
setTimeout(function() {
jQuery(".maineditor21806 .copymessage").hide();
}, 2000);
}).catch(err => {
console.error("Error cloning code: ", err);
});
}
function runCode21806() {
var code = editor21806.getSession().getValue();
function closeoutput21806() {
var code = editor21806.getSession().getValue();
jQuery(".maineditor21806 .code-editor-output").hide();
}
// Attach event listeners to the buttons
document.getElementById("copyBtn21806").addEventListener("click", copyCodeToClipboard21806);
document.getElementById("runBtn21806").addEventListener("click", runCode21806);
document.getElementById("closeoutputBtn21806").addEventListener("click", closeoutput21806);
Outcome:
Analysis:
The preceding code is employed to visualize the trained Decision Tree Classifier utilizing plot_tree(). It aids in displaying feature names, class names, and color-filled nodes in a matplotlib plot.
Step 3: Extracting Decision Trees in Written Format
Now, let’s discuss the extraction of the decision rules in a format that is easily understandable.
Illustration:
Python
Code Copied!
var isMobile = window.innerWidth
``````javascript
");
editor67777.setValue(decodedContent); // Establish the default text
editor67777.clearSelection();
editor67777.setOptions({
maxLines: Infinity
});
function decodeHTML67777(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to duplicate code to clipboard
function copyCodeToClipboard67777() {
const code = editor67777.getValue(); // Obtain code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code successfully copied to clipboard!");
function closeoutput67777() {
var code = editor67777.getSession().getValue();
jQuery(".maineditor67777 .code-editor-output").hide();
}
// Bind event listeners to the buttons
document.getElementById("copyBtn67777").addEventListener("click", copyCodeToClipboard67777);
document.getElementById("runBtn67777").addEventListener("click", runCode67777);
document.getElementById("closeoutputBtn67777").addEventListener("click", closeoutput67777);
Result:
Clarification:
The above snippet is utilized for retrieving and displaying the human-readable decision rules of the trained Decision Tree Classifier. It employs export_text(), and also incorporates feature names.
Step 4: Retrieving Decision Rules as Python Code
To extract rules in Python’s if-else pattern, Scikit-learn offers a method to transform the tree into a Python script.
Illustration:
Python
Code Successfully Copied!
var isMobile = window.innerWidth ");
editor35953.setValue(decodedContent); // Establish the default text
editor35953.clearSelection();
editor35953.setOptions({
maxLines: Infinity
});
function decodeHTML35953(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to duplicate code to clipboard
function copyCodeToClipboard35953() {
const code = editor35953.getValue(); // Obtain code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code successfully copied to clipboard!");
function closeoutput35953() {
var code = editor35953.getSession().getValue();
jQuery(".maineditor35953 .code-editor-output").hide();
}
``````javascript
// Attach event listeners to the buttons
document.getElementById("copyBtn35953").addEventListener("click", copyCodeToClipboard35953);
document.getElementById("runBtn35953").addEventListener("click", runCode35953);
document.getElementById("closeoutputBtn35953").addEventListener("click", closeoutput35953);
Output:
Explanation:
The code presented above is utilized to recursively obtain and display the decision rules from a trained Decision Tree Classifier. This facilitates showing conditions based on feature thresholds along with their respective outputs.
Step 5: Transforming Rules into a Pandas DataFrame
To achieve a more organized representation, we will extract the decision rules into a Pandas DataFrame.
Example:
Python
Code Copied!
var isMobile = window.innerWidth ");
editor94687.setValue(decodedContent); // Set the default text
editor94687.clearSelection();
editor94687.setOptions({
maxLines: Infinity
});
function decodeHTML94687(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to copy code to clipboard
function copyCodeToClipboard94687() {
const code = editor94687.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code copied to clipboard!");
function closeoutput94687() {
var code = editor94687.getSession().getValue();
jQuery(".maineditor94687 .code-editor-output").hide();
}
// Attach event listeners to the buttons
document.getElementById("copyBtn94687").addEventListener("click", copyCodeToClipboard94687);
document.getElementById("runBtn94687").addEventListener("click", runCode94687);
document.getElementById("closeoutputBtn94687").addEventListener("click", closeoutput94687);
Output:
Explanation:
The code above is utilized to extract decision rules from a trained Decision Tree Classifier. It subsequently formats them as logical conditions and preserves them in a DataFrame. This approach effectively illustrates the distribution of classes based on the defined rules.
Visualizing Decision Boundaries of a Decision Tree
Understanding how a decision tree segments the feature space is crucial as it aids in interpreting the model’s performance. To visualize the decision boundaries for a dataset, we can focus on two features.
Example:
Python
``````html
Code Copied!
var isMobile = window.innerWidth ");
editor73793.setValue(decodedContent); // Assign the default text
editor73793.clearSelection();
editor73793.setOptions({
maxLines: Infinity
});
function decodeHTML73793(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to copy code to clipboard
function copyCodeToClipboard73793() {
const code = editor73793.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code copied to clipboard!");
function closeoutput73793() {
var code = editor73793.getSession().getValue();
jQuery(".maineditor73793 .code-editor-output").hide();
}
// Attach event listeners to the buttons
document.getElementById("copyBtn73793").addEventListener("click", copyCodeToClipboard73793);
document.getElementById("runBtn73793").addEventListener("click", runCode73793);
document.getElementById("closeoutputBtn73793").addEventListener("click", closeoutput73793);
Output:
Explanation:
The preceding code is utilized to train a Decision Tree Classifier on two attributes of the Iris dataset. It also illustrates its decision boundaries through a contour plot.
Feature Significance in Decision Trees
Decision trees assist in revealing which attributes are most pivotal in forming predictions.
Example:
Python
Code Copied!
var isMobile = window.innerWidth ");
editor28720.setValue(decodedContent); // Assign the default text
editor28720.clearSelection();
editor28720.setOptions({
maxLines: Infinity
});
function decodeHTML28720(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to copy code to clipboard
function copyCodeToClipboard28720() {
const code = editor28720.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code copied to clipboard!");
function hideOutput28720() {
var code = editor28720.getSession().getValue();
jQuery(".maineditor28720 .code-editor-output").hide();
}
// Bind event handlers to the buttons
document.getElementById("copyBtn28720").addEventListener("click", copyCodeToClipboard28720);
document.getElementById("runBtn28720").addEventListener("click", executeCode28720);
document.getElementById("closeoutputBtn28720").addEventListener("click", hideOutput28720);
Output:
Clarification:
The code presented above is utilized to derive and display the feature importance scores from the trained Decision Tree classifier. This is executed for the initial two features of the Iris dataset.
Tuning Hyperparameters for Decision Trees
Fine-tuning the parameters of the tree like max_depth, min_samples_split, and min_samples_leaf enhances the effectiveness of the model.
Illustration:
Python
Code Successfully Copied!
var isMobile = window.innerWidth ");
editor91974.setValue(decodedContent); // Assign the default text
editor91974.clearSelection();
editor91974.setOptions({
maxLines: Infinity
});
function decodeHTML91974(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to copy code to clipboard
function copyCodeToClipboard91974() {
const code = editor91974.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code duplicated to clipboard!");
function hideOutput91974() {
var code = editor91974.getSession().getValue();
jQuery(".maineditor91974 .code-editor-output").hide();
}
// Bind event handlers to the buttons
document.getElementById("copyBtn91974").addEventListener("click", copyCodeToClipboard91974);
document.getElementById("runBtn91974").addEventListener("click", executeCode91974);
document.getElementById("closeoutputBtn91974").addEventListener("click", hideOutput91974);
Output:
Clarification:
The code above conducts hyperparameter optimization on a Decision Tree classifier. It employs GridSearchCV utilizing 5-fold cross-validation, examining different values for max_depth,min_samples_split, and min_samples_leaf. It subsequently displays the optimal set of parameters.
Conclusion
This blog post delves into assorted methodologies for engaging with decision trees within Scikit-Learn. This encompasses visualizing decision boundaries, comprehending feature importance, fine-tuning hyperparameters, extracting decision rules, and preserving them in structured formats. Such methodologies enhance interpretability and facilitate the smooth integration of decision trees into diverse machine learning workflows.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
