[T-SQL]An arithmetic overflow error

Solved
pinko_01 Posted messages 2 Registration date   Status Member Last intervention   -  
pinko_01 Posted messages 2 Registration date   Status Member Last intervention   -
Hello,
I'm working with stored procedures on SQL Server 2008, but when I try to execute one of my procedures, I get the following error:

An arithmetic overflow error occurred during the conversion of an expression to data type int.

[CODE]
USE [Entreprise]
GO
/****** Object: StoredProcedure [dbo].[Cos_phi_calculé] Script Date: 06/12/2010 11:21:50 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER procedure [dbo].[Cos_phi_calculé]
as
select dbo.CosPhi_calcule(consomm_HC,consomm_HP,consomm_HN,consomm_energie_reactive)
from dbo.Facture
[/CODE]

Indeed, I have a function called [dbo].[CosPhi_calcule] that returns a value of type float. When I try to retrieve it with my stored procedure [dbo].[Cos_phi_calculé], I get the error previously mentioned.

[CODE]

ALTER FUNCTION [dbo].[CosPhi_calcule]
(
@consomm_HC int,
@consomm_HP int,
@consomm_HN int,
@consoER int
)
RETURNS float
AS
BEGIN
declare @resultat float

set @resultat=(@consomm_HC+@consomm_HP+@consomm_HN)/(SQRT(((@consomm_HC+@consomm_HP+@consomm_HN)*(@consomm_HC+@consomm_HP+@consomm_HN)) + (@consoER * @consoER)))
return @resultat
END

[/CODE]

In the parameters of my procedure, I find this:

[IMG]http://www.easy-upload.net/fichier.php?fichier=2010612123640[/IMG]

As you can see, my procedure always returns an integer type value, and I think that's the only reason why I get the overflow message.

So I modified the return type of my function and set it to int instead of float so that my procedure could capture the value of the function, but unfortunately in vain, I still get the same error.

I also have this code:

[CODE]
USE [Entreprise]
GO

[B]DECLARE @return_value int[/B]

EXEC @return_value = [dbo].[Cos_phi_calculé]

SELECT 'Return Value' = @return_value

GO

[/CODE]

Here as well, I replaced int with float to remedy the conversion problem, but to no avail.

So is there a way to change the return type of my procedure, making it float instead of int, because int is set by default by SQL Server?

I hope I explained my problem well and your comments will be welcome.

Thank you in advance :)

2 answers

fiu
 
Hi,

I don’t know the intricacies of SQL Server, but reading your code reminded me of some unpleasant experiences.
Since the SQRT function returns a float, it’s better to provide it with floats.

So I would tend to advise you to explore the following approach:
* explicitly convert all ints to floats
* detail each calculation
* convert the final result back to int
If you manage to pinpoint the source of the bug, you can then save on conversions and only convert the sensitive part.
1