During the development of machine learning models in Keras, Mean Squared Error (MSE) is frequently employed as the loss function for regression tasks. In contrast to classification, regression concerns continuous values, rendering standard accuracy measurements inapplicable. Since MSE concentrates on reducing errors instead of predicting distinct labels, Keras does not furnish accuracy metrics for regression endeavors. Rather, alternatives such as Mean Absolute Error (MAE), R^2 Score, and Mean Absolute Percentage Error (MAPE) are utilized. Nevertheless, a personalized accuracy function can be formulated, which will aid in assessing predictions within an acceptable error margin. This offers an accuracy-like evaluation for regression models.
This article will delve into Mean Squared Error, the formulation of an accuracy function for regression, and the way Keras defines accuracy when utilizing MSE as the loss function. So let’s begin!
Mean Squared Error (MSE) is generally applied to regression assignments. It measures the disparity between predicted values and actual values.
The formula for MSE (Mean Squared Error) is provided below:
Where,
y true represents the actual values.
y pred indicates the predicted values.
n signifies the total number of samples.
A low MSE indicates that the model’s predictions are closer to the true values.
Does Accuracy Function with MSE?
In classification scenarios, accuracy is a clear-cut concept. It measures the number of predictions that correspond with the actual labels. Accuracy can be articulated as:
Accuracy = Correct Predictions / Total Predictions
The notions of “correct” or “incorrect” predictions do not apply in regression tasks, where the aim is to forecast continuous values. Instead of binary correctness, predictions are assessed by their proximity to actual values. Therefore, accuracy is not an appropriate metric for regression tasks.
MSE (Mean Squared Error) quantifies the average squared differences between the predicted and actual values. As a result, it does not concentrate on tallying correct predictions; rather, it is focused on minimizing the overall error. Owing to this distinction, accuracy is not preset as a metric by Keras when MSE is employed as the loss function. Instead, alternative evaluation metrics like Mean Absolute Error (MAE), R^2 Score, and Mean Absolute Percentage Error (MAPE) are utilized to evaluate model performance in regression contexts.
Creating an Accuracy Function for Regression
Keras does not include a built-in metric for accuracy with MSE. Hence, you can establish your personalized accuracy function to assess accuracy while applying Mean Squared Error (MSE) as the loss function. Unlike classification, where accuracy is unambiguous, regression accuracy assesses how closely the predicted values align with the actual values.
1. Employing Mean Absolute Percentage Error (MAPE)
Accuracy in regression can be gauged using Mean Absolute Percentage Error (MAPE). It calculates the average percentage error between actual and predicted values. The formula for MAPE is presented below:
MAPE articulates the error as a percentage, making it a beneficial measure.metric for assessing the correspondence of the model’s forecasts with the actual values. A lower MAPE indicates enhanced model performance.
2. Creating a Custom Accuracy Function in Keras
Alternatively, one can establish a custom accuracy function that treats a prediction as “correct” even when it lies within a designated tolerance level of the actual value. For instance, you may consider the prediction valid if the anticipated data is within a 10% deviation from the actual value.
This method aids in converting the regression output of the model into a metric resembling accuracy. This facilitates the interpretation of the model’s efficacy in contexts where absolute precision is not feasible. By incorporating these functions in Keras, you can more effectively monitor the advancements of the model for regression applications.
Execution of Custom Accuracy in Keras
We will proceed to train a fundamental Keras Regression Model and subsequently define a custom accuracy function. Below are the outlined steps for its execution.
Step 1: Importing the Required Libraries
Example:
Python
Code Copied!
var isMobile = window.innerWidth “);
editor89028.setValue(decodedContent); // Set the default text
editor89028.clearSelection();
editor89028.setOptions({
maxLines: Infinity
});
function decodeHTML89028(input) {
var doc = new DOMParser().parseFromString(input, “text/html”);
return doc.documentElement.textContent;
}
// Function to copy code to clipboard
function copyCodeToClipboard89028() {
const code = editor89028.getValue(); // Get code from the editor
navigator.clipboard.writeText(code).then(() => {
jQuery(“.maineditor89028 .copymessage”).show();
setTimeout(function() {
jQuery(“.maineditor89028 .copymessage”).hide();
}, 2000);
}).catch(err => {
console.error(“Error copying code: “, err);
});
}
function runCode89028() {
var code = editor89028.getSession().getValue();
function closeoutput89028() {
var code = editor89028.getSession().getValue();
jQuery(“.maineditor89028 .code-editor-output”).hide();
}
// Attach event listeners to the buttons
document.getElementById(“copyBtn89028”).addEventListener(“click”, copyCodeToClipboard89028);
document.getElementById(“runBtn89028”).addEventListener(“click”, runCode89028);
document.getElementById(“closeoutputBtn89028”).addEventListener(“click”, closeoutput89028);
Clarification:
The preceding section solely imports the requisite libraries such as NumPy, TensorFlow/Keras, and Scikit-Learn needed to build and train the neural network. This snippet will not produce any output since it simply contains the import statements and does not define or execute any functions, models, or data manipulation procedures.
Step 2: Creating a Sample Dataset
In this step, we will construct a straightforward dataset where y = 3x + 5.
Example:
Python
Code Copied!
var isMobile = window.innerWidth “);
editor61244.setValue(decodedContent);“`javascript
// Clear the default text
editor61244.clearSelection();
editor61244.setOptions({
maxLines: Infinity
});
function decodeHTML61244(input) {
var doc = new DOMParser().parseFromString(input, “text/html”);
return doc.documentElement.textContent;
}
// Function to duplicate code to clipboard
function copyCodeToClipboard61244() {
const code = editor61244.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert(“Code copied to clipboard!”);
function closeOutput61244() {
var code = editor61244.getSession().getValue();
jQuery(“.maineditor61244 .code-editor-output”).hide();
}
// Attach event listeners to the buttons
document.getElementById(“copyBtn61244”).addEventListener(“click”, copyCodeToClipboard61244);
document.getElementById(“runBtn61244”).addEventListener(“click”, runCode61244);
document.getElementById(“closeoutputBtn61244”).addEventListener(“click”, closeOutput61244);
Clarification:
The code above is designed for generating artificial data. It employs a linear equation with added noise, subsequently dividing the data into training and testing sets following the 80:20 distribution. This code does not produce any output since it solely splits the data without inclusive print or visualization commands to exhibit the generated values.
Step 3: Crafting a Custom Accuracy Function
At this point, we will outline an accuracy function to ascertain the count of predictions that fall within 10% of the actual value.
Illustration:
Python
Code Copied!
var isMobile = window.innerWidth “);
editor55150.setValue(decodedContent); // Set the default text
editor55150.clearSelection();
editor55150.setOptions({
maxLines: Infinity
});
function decodeHTML55150(input) {
var doc = new DOMParser().parseFromString(input, “text/html”);
return doc.documentElement.textContent;
}
// Function to duplicate code to clipboard
function copyCodeToClipboard55150() {
const code = editor55150.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert(“Code copied to clipboard!”);
function closeOutput55150() {
var code = editor55150.getSession().getValue();
jQuery(“.maineditor55150 .code-editor-output”).hide();
}
// Attach event listeners to the buttons
document.getElementById(“copyBtn55150”).addEventListener(“click”, copyCodeToClipboard55150);
document.getElementById(“runBtn55150”).addEventListener(“click”, runCode55150);
document.getElementById(“closeoutputBtn55150”).addEventListener(“click”, closeOutput55150);
Clarification:
In the preceding code, the function custom_accuracy serves to determine the ratio of predictions that fall within 10% of the true values. This is achieved through the assessment of the relative error and the calculation of the mean for the correct predictions. The code does not produce any output as it merely defines a function without executing any data inputs.
Step 4: Developing the Regression Model
At this stage, we will assemble a basic neural network utilizing Mean Squared Error along with our bespoke loss accuracy function.
Illustration:
“““html
class=”main-editor-wrapper”>
Code Duplicated!
var isMobile = window.innerWidth “);
editor57199.setValue(decodedContent); // Assign the default text
editor57199.clearSelection();
editor57199.setOptions({
maxLines: Infinity
});
function decodeHTML57199(input) {
var doc = new DOMParser().parseFromString(input, “text/html”);
return doc.documentElement.textContent;
}
// Function to replicate code to clipboard
function copyCodeToClipboard57199() {
const code = editor57199.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert(“Code replicated to clipboard!”);
function closeoutput57199() {
var code = editor57199.getSession().getValue();
jQuery(".maineditor57199 .code-editor-output").hide();
}
// Attach event listeners to the buttons
document.getElementById("copyBtn57199").addEventListener("click", copyCodeToClipboard57199);
document.getElementById("runBtn57199").addEventListener("click", runCode57199);
document.getElementById("closeoutputBtn57199").addEventListener("click", closeoutput57199);
Clarification:
This code snippet is utilized to define a neural network featuring two concealed layers. Subsequently, it compiles it using the Adam Optimizer alongside Mean Squared Error (MSE) loss function. It trains the model on the dataset while monitoring a custom accuracy measure. The code does not produce an output as the model's training function ( model.fit ) operates in silence ( verbose = 0 ). Consequently, no logs or progress notifications are displayed.
Step 5: Model Assessment
In this phase, we will assess the model's performance.
Sample:
Python
Code Duplicated!
var isMobile = window.innerWidth ");
editor70433.setValue(decodedContent); // Assign the default text
editor70433.clearSelection();
editor70433.setOptions({
maxLines: Infinity
});
function decodeHTML70433(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Function to replicate code to clipboard
function copyCodeToClipboard70433() {
const code = editor70433.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code replicated to clipboard!");
function closeoutput70433() {
var code = editor70433.getSession().getValue();
jQuery("e;.maineditor70433 .code-editor-output"e;).hide();
}
// Bind event listeners to the buttons
document.getElementById("e;copyBtn70433"e;).addEventListener("e;click"e;, copyCodeToClipboard70433);
document.getElementById("e;runBtn70433"e;).addEventListener("e;click"e;, runCode70433);
document.getElementById("e;closeoutputBtn70433"e;).addEventListener("e;click"e;, closeoutput70433);
Output:
Analysis:
The output above indicates that the trained model has an MSE loss of 4.3449 on the test dataset, accompanied by a custom accuracy metric of 0.6641. This signifies that 66.41% of the predictions were within the established accuracy range.
Alternative Metrics for Assessing Regression Accuracy
For different methods of evaluating accuracy in regression, consider the following alternatives:
1. R^2 Score (Coefficient of Determination)
This metric is utilized to assess how closely the predicted values align with the actual outcomes. A significant discrepancy indicates better performance.
Example:
Python
Code Copied!
var isMobile = window.innerWidth {
// alert("e;Code copied to clipboard!"e;);
function closeoutput17596() {
var code = editor17596.getSession().getValue();
jQuery("e;.maineditor17596 .code-editor-output"e;).hide();
}
// Bind event listeners to the buttons
document.getElementById("e;copyBtn17596"e;).addEventListener("e;click"e;, copyCodeToClipboard17596);
document.getElementById("e;runBtn17596"e;).addEventListener("e;click"e;, runCode17596);
document.getElementById("e;closeoutputBtn17596"e;).addEventListener("e;click"e;, closeoutput17596);
Output:
Analysis:
The above illustrates the calculation of the R^2 (coefficient of determination) score. This metric evaluates the model's performance by correlating actual and predicted values, subsequently outputting the results rounded to four decimal places.
2. Mean Absolute Error (MAE)
This metric quantifies the average absolute disparity between the actual and estimated values.
Example:
Python
```
var isMobile = window.innerWidth ");
editor74011.setValue(decodedContent); // Establish the standard text
editor74011.clearSelection();
editor74011.setOptions({
maxLines: Infinity
});
function decodeHTML74011(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
// Procedure to copy the code to clipboard
function copyCodeToClipboard74011() {
const code = editor74011.getValue(); // Retrieve code from the editor
navigator.clipboard.writeText(code).then(() => {
// alert("Code successfully copied to clipboard!");
function closeOutput74011() {
var code = editor74011.getSession().getValue();
jQuery(".maineditor74011 .code-editor-output").hide();
}
// Bind event listeners to the buttons
document.getElementById("copyBtn74011").addEventListener("click", copyCodeToClipboard74011);
document.getElementById("runBtn74011").addEventListener("click", executeCode74011);
document.getElementById("closeoutputBtn74011").addEventListener("click", closeOutput74011);
Result:
Clarification:
The code above computes the Mean Absolute Error (MAE) between the actual and predicted values. This quantifies the average prediction deviation of the model. The result is subsequently displayed, rounded to four decimal digits.
Summary
When utilizing Mean Squared Error (MSE) as a loss function in Keras, conventional accuracy metrics are inapplicable. This is due to the nature of regression tasks, which lack a definitive “correct” or “incorrect” prediction framework. Instead, one may create tailored accuracy metrics such as Mean Absolute Percentage Error (MAPE) or an accuracy measure based on tolerances, which will provide meaningful evaluations of the model. Additionally, metrics such as R^2 score and Mean Absolute Error (MAE) contribute valuable insights into the model's predictive effectiveness. By selecting appropriate assessment methods, one can enhance the evaluation and refinement of accuracy for the regression model in Keras.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
