Originally posted at http://www.howtomeasureanything.com, on Sunday, September 06, 2009 9:20:07 PM, by sujoymitra17.
“While using Lens Model (multiple regression), I am getting negative scores for a few parameters and positive scores for few others. I am computing the score using the formula:
– <Coeff of parameter-1>*Val of parameter-1+<Coeff of parameter-2>*Val of parameter-2….+Intercept.
Since few parameters are showing -ve scores and others +ve (considering few have -ve correlation-coeff and others have +ve correlation-coeff), how do I formulate weights?”
I’m a little confused by your message. The coefficients in a regression model ARE the “weights”. The output of a regression analysis includes the coefficients. A regression analysis is how the weights are computed for a Lens model (the former is a tool for the later, they are not the same thing).
Are you actually performing a least-squares best-fit linear regression analysis? Are you using the regression tool in Excel? Just making a formula with parameters and coefficients is not a linear regression.
Getting negative coefficients is not necessarily a problem, since that actually makes sense for many situations (examples of negative coefficients include criminal convictions and income, body fat and life expectancy, driving speed and mileage, etc.) If you are doing an actual regression, then getting negative values is not a problem. It can even be an expected outcome.
Perhaps you can describe what you are attempting to do in more detail.
Thanks,
Doug Hubbard
Hi Doug,
Thanks for the prompt response.
I am trying to evaluate the business criticality of various BUs based on the criticality of the IT apps. In doing so, I have considered a few factors like Accessibility of the apps, App’s functionality, Extent of client access, etc. I am treating them as independent variables(X). My dependent variable is business criticality(Y). Now I have applied 30 different scenarios for the independent variables. For each, I have a calibrated the business criticality of the apps. When I do a regression (using excel), I find a -ve intercept (Y is -ve when X is 0) and a mix of +ve and -ve coefficients. Now I am categorizing the apps under each BU. For say BU-1, I am computing the weight using the formula: – (coeff of Accessibility*% of apps under BU-1 that are accessible globally)+(Coeff of client access*% of apps under BU-1 that are accessible by clients)+…+(Intercept). I have the following result: –
BU-1: 0.18
BU-2: -2.14
BU-3: 3.22, etc.
Now, since my objective is to compute the business criticality of each BU (from an IT apps perspective), I need to have all +ve weights.
I am using excel for regression. I am using line-fit plots to generate my graph and hence I assume I am following linear least square best-fit regression model. DO I consider my Y intercept as 0 always (by choosing constant=0 while doing regression in excel), so that all my weights are +ve? Alternatively, do you see a flaw in my understanding of Lens? Looking at your response I guess you are saying that the coeff itself should be treated as weights. In that case, do let me know the right approach.
“Business criticality” seems like a fuzzy concept and that might be a source of confusion. What objective outcomes are you actually forecasting? For example, if you were building this model with historical data instead of a Lens, what would the historical data for “Business criticality” be?
You said you are computing a “weight” using a formula which is actually itself the sum of a set of weights (coefficients) times independent variables. It appears that what you were calling the “weight” is actually your dependent variable Y, is that correct? If so, what is this weight used for? I’m still not clear what it is your model is doing.
If I knew what “business criticality” actually meant and what you use it for then we could talk about whether it makes sense for some variables to have negative coefficients or for the dependent variable to be negative. If business criticality is just a relative priority, then why not be negative? What is the definition of business criticality (in terms of objective observations) that would make a negative value nonsensical?
If, for some reason, you think coefficients or Y-estimates should not be negative, then we would have to get further into your data to see why you are getting that result. Perhaps you could share your scenario data. Feel free to send it to me at dwhubbard@hubbardresearch.com.
Thanks,
Doug
Actually let me restate Y. Y is the criticality of IT Apps in a Business Unit. Thus, if I have 3 business units, then ideally I want to find the following: –
BU-1: 0.25 (Implies 0.25 is the criticality of IT apps in BU-1)
BU-2: 0.47 (Implies that 0.47 is the criticality of IT apps in BU-2)…etc. My objective is to find the logic behind this weight and hence I am trying to use Lens Model.
Thus, in the first level, for the entire company, I have taken 30 different samples of apps irrespective of BU. For each I have also calibrated the apps in the sample. The factors I have considered to calibrate are: –
1.Business Process being supported by most of the apps in the sample
2.Functionality of the apps in the sample
After regression, the Multiple-R is 0.78 and R-sq is 0.58
For this I have the following weights:
% Global 18.13483736
% Client Access 3.021636144
% Partner Access -16.79569036
Intercept -3.108299825
Ideally both client and partner access should have +ve correlation, because if an application is accessed by client or by partners/suppliers, it ought to be an important app. But here we find a -ve correlation.
Now when we have the regression for all the apps in the company, I am trying to find the criticality of apps in each BU. For that I categorize apps under each BU. Say for BU-1, we have the following result: –
% of Global Apps in BU-1 = 3%
% of client access = 2%
% of partner access = 1%
Then value of BU-1 = 3%X18.13+2%X3.02+1%X(-16.79)-3.10=-2.67. Similarly, I am computing the criticality of apps in BU-2 which is +ve. I agree that the weights are relative and hence -ve might make sense, but this part of the weight assignment is part of a bigger model I am working on (I cannot reveal owing to IP restrictions). But, it being weights that will be used for further calculation, I need to have all positive weights. What should I do? If you are yet unable to comprehend my problem, I can try sending you some dummy data for illustration.
The problem I’m having is still the one I had before. I still don’t know what you mean by “criticality”. Is this some term commonly used in your field to mean something specific? (My only knowledge of a precise definition of this term applies to critical masses in nuclear physics, which I’m sure you aren’t talking about). Perhaps you can answer some of the questions I pose in the book.
-Why do you care about “criticality”? What decision could be influenced by this measure?
-What would you do differently if it turned out to be higher than expected? What if it were lower than expected?
-How do you observe it if it changes?
-What is the expected range of possible “criticalities”?
It seems to me like you are attempting to use a Lens model to evaluate some invented arbitrary prioritization score. I don’t inherently see a problem with that but I suspect its not an ideal application of the tool. Perhaps we need to back up a level and answer these general scope questions about the term. Without understanding what this term really means I don’t know whether it makes sense or not to have a -ve or even negative Y value. Of course, you can always renormalize the output by redefining the minimum criticality as 0 and the maximum value as 1 or whatever. But that doesn’t help us determine if you have some math error that causes you to get negative values when you really shouldn’t.