Is there any solution for this
Thanks
Niru
Findout out the solution...using Sqlquery as OLEDB Source instead of Table.This solved the thing.
Thanks
Niru
sql
Thanks
Niru
Findout out the solution...using Sqlquery as OLEDB Source instead of Table.This solved the thing.
Thanks
Niru
sql
Hi all;
I have created a SSIS package to import data from flat file to database table, which consists of
Flat file Source -> Look up -> OLE DB Destination.
I'm using Look Up in order to avoid duplicate records.
My database table contain around 35,00000 s of rows.
So when I'm running the package the above amount of data goes to Lookup cache, which require a large storage memory.
So I'm not able to execute my package.
I need to run the package in order to update my database.
So is there any solution, in which I can check the duplicate records to get into the database and time consumption to run the package will be less.
Thanks in advance...
select primary_key from table
Then you just simply join the key fields.|||
Hi Sanidha,
You may want to consider key staging the lookup table. Prior to performing the lookup against a 35 million row table, create a staging table containing only the columns (and rows, if identifiable) needed to perform the lookup, then use it instead of the full table.
Hope this helps,
Andy
Andy Leonard wrote:
Hi Sanidha,
You may want to consider key staging the lookup table. Prior to performing the lookup against a 35 million row table, create a staging table containing only the columns (and rows, if identifiable) needed to perform the lookup, then use it instead of the full table.
Hope this helps,
Andy
Why would this matter? Selecting the primary key of the full table (should be indexed!) should be less work than staging the keys first, and then selecting from there. That sounds like an extra, unnecessary step to me and will, in turn, cost more than my approach.
Am I missing something? Unless of course, you're talking about performing a SQL join against that staging table so as to avoid caching the keys in memory.|||
My apologies Sanidha,
I thought the lookup table contained too many rows to fully cache - it was an assumption on my part.
Andy
|||Andy Leonard wrote:
My apologies Sanidha,
I thought the lookup table contained too many rows to fully cache - it was an assumption on my part.
Andy
Well, yeah, that's the one thing we don't know. I want to be sure the user is selecting *just* the key, and not the whole table.
below is my problem. Let's say I have 2 tables, a Products table and a
Colors table that go as follow:
Table Products
prodID Name
1 shirt
2 tshirt
3 pants
4 jeans
and
Table Colors
prodID Colors
1 Blue
1 Red
2 Blue
3 Black
3 White
4 Blue
I want to find out all the products that come in Blue, and if not I
want to have the color empty. The result I want from my Query / SQL
Statement is:
prodID Colors Name...
1 Blue
2 Blue
3
4 Blue
What should my SQL statement/Query be like?
I tried:
Select Product.ProdID, Colors.Colors
From
Products
Left Join Colors
on Product.ProdID = Colors.ProdID
where Colors.Colors = "blue"
and this is what I get:
prodID Colors
1 Blue
2 Blue
4 Blue
Notice that prodID 2 doesn't show up but I want to return all prodIDs
whether or not they have a color Blue.
Please help...
Thanks[posted and mailed, please reply in news]
Allan (proflicker@.hotmail.com) writes:
> Select Product.ProdID, Colors.Colors
> From
> Products
> Left Join Colors
> on Product.ProdID = Colors.ProdID
> where Colors.Colors = "blue"
When you say:
FROM a LEFT JOIN b on ...
You are, concpetually, constructing a table. Then you apply a WHERE
clause to filter out rows from that table.
Thus for
Products Left Join Colors on Product.ProdID = Colors.ProdID
You get a table with data in all columns for Products, but where
there is no matching row in Colors, you get NULL.
Then you apply a WHERE clause to this, but then you filter all those
NULL rows, because NULL is not equal to "blue".
The remedy is to move the condition to the ON clause:
Products Left Join Colors
on Product.ProdID = Colors.ProdID
and Colors.Color = "blue"
Now the condition on Colors becomes part of that conceptual table.
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||On 7 Jun 2004 15:26:39 -0700, Allan wrote:
>Please help,
>below is my problem. Let's say I have 2 tables, a Products table and a
>Colors table that go as follow:
>Table Products
>prodID Name
>1 shirt
>2 tshirt
>3 pants
>4 jeans
>and
>Table Colors
>prodID Colors
>1 Blue
>1 Red
>2 Blue
>3 Black
>3 White
>4 Blue
>
>I want to find out all the products that come in Blue, and if not I
>want to have the color empty. The result I want from my Query / SQL
>Statement is:
>prodID Colors Name...
>1 Blue
>2 Blue
>3
>4 Blue
>What should my SQL statement/Query be like?
>I tried:
>Select Product.ProdID, Colors.Colors
>From
>Products
>Left Join Colors
>on Product.ProdID = Colors.ProdID
>where Colors.Colors = "blue"
>and this is what I get:
>prodID Colors
>1 Blue
>2 Blue
>4 Blue
>
>Notice that prodID 2 doesn't show up but I want to return all prodIDs
>whether or not they have a color Blue.
>Please help...
>Thanks
Hi Allan,
Try:
Select Product.ProdID, Colors.Colors
From
Products
Left Join Colors
on Product.ProdID = Colors.ProdID
and Colors.Colors = 'blue'
(untested)
Note: Changed "where" to "and" and also changed double-quotes to
single-quotes (single quotes are the standard string delimiter for SQL, as
defined by the ANSI standard. Double quotes are, depending on the setting
of some option, still supported in SQL Server 2000 for backward
compatibility)
Best, Hugo
--
(Remove _NO_ and _SPAM_ to get my e-mail address)|||Thanks for the quick reply...
I had already tried that and I get an error message saying:
JOIN expression not supported
Any other suggestions?
Gad
Hugo Kornelis <hugo@.pe_NO_rFact.in_SPAM_fo> wrote in message news:<94r9c0lovk9hshhb5monp75s24mg1abnd2@.4ax.com>...
> On 7 Jun 2004 15:26:39 -0700, Allan wrote:
> >Please help,
> >below is my problem. Let's say I have 2 tables, a Products table and a
> >Colors table that go as follow:
> >Table Products
> >prodID Name
> >1 shirt
> >2 tshirt
> >3 pants
> >4 jeans
> >and
> >Table Colors
> >prodID Colors
> >1 Blue
> >1 Red
> >2 Blue
> >3 Black
> >3 White
> >4 Blue
> >I want to find out all the products that come in Blue, and if not I
> >want to have the color empty. The result I want from my Query / SQL
> >Statement is:
> >prodID Colors Name...
> >1 Blue
> >2 Blue
> >3
> >4 Blue
> >What should my SQL statement/Query be like?
> >I tried:
> >Select Product.ProdID, Colors.Colors
> >From
> >Products
> >Left Join Colors
> >on Product.ProdID = Colors.ProdID
> >where Colors.Colors = "blue"
> >and this is what I get:
> >prodID Colors
> >1 Blue
> >2 Blue
> >4 Blue
> >Notice that prodID 2 doesn't show up but I want to return all prodIDs
> >whether or not they have a color Blue.
> >Please help...
> >Thanks
> Hi Allan,
> Try:
> Select Product.ProdID, Colors.Colors
> From
> Products
> Left Join Colors
> on Product.ProdID = Colors.ProdID
> and Colors.Colors = 'blue'
> (untested)
> Note: Changed "where" to "and" and also changed double-quotes to
> single-quotes (single quotes are the standard string delimiter for SQL, as
> defined by the ANSI standard. Double quotes are, depending on the setting
> of some option, still supported in SQL Server 2000 for backward
> compatibility)
> Best, Hugo|||"Allan" <proflicker@.hotmail.com> wrote in message
news:7b5b0602.0406071919.319fa873@.posting.google.c om...
> Thanks for the quick reply...
> I had already tried that and I get an error message saying:
> JOIN expression not supported
> Any other suggestions?
The name of your products table is "Products", right?
You have "Product" in the join condition of your query.
SELECT P.prodID, C.colors
FROM Products AS P
LEFT OUTER JOIN
Colors AS C
P.prodID = C.prodID AND
C.colors = 'blue'
--
JAG
> Gad
>
> Hugo Kornelis <hugo@.pe_NO_rFact.in_SPAM_fo> wrote in message
news:<94r9c0lovk9hshhb5monp75s24mg1abnd2@.4ax.com>...
> > On 7 Jun 2004 15:26:39 -0700, Allan wrote:
> > >Please help,
> > > >below is my problem. Let's say I have 2 tables, a Products table and a
> > >Colors table that go as follow:
> > > >Table Products
> > > >prodID Name
> > >1 shirt
> > >2 tshirt
> > >3 pants
> > >4 jeans
> > > >and
> > > >Table Colors
> > > >prodID Colors
> > >1 Blue
> > >1 Red
> > >2 Blue
> > >3 Black
> > >3 White
> > >4 Blue
> > > > >I want to find out all the products that come in Blue, and if not I
> > >want to have the color empty. The result I want from my Query / SQL
> > >Statement is:
> > > >prodID Colors Name...
> > > >1 Blue
> > >2 Blue
> > >3
> > >4 Blue
> > > >What should my SQL statement/Query be like?
> > > >I tried:
> > > >Select Product.ProdID, Colors.Colors
> > >From
> > >Products
> > >Left Join Colors
> > >on Product.ProdID = Colors.ProdID
> > >where Colors.Colors = "blue"
> > > >and this is what I get:
> > > >prodID Colors
> > > >1 Blue
> > >2 Blue
> > >4 Blue
> > > > >Notice that prodID 2 doesn't show up but I want to return all prodIDs
> > >whether or not they have a color Blue.
> > > >Please help...
> > > >Thanks
> > Hi Allan,
> > Try:
> > Select Product.ProdID, Colors.Colors
> > From
> > Products
> > Left Join Colors
> > on Product.ProdID = Colors.ProdID
> > and Colors.Colors = 'blue'
> > (untested)
> > Note: Changed "where" to "and" and also changed double-quotes to
> > single-quotes (single quotes are the standard string delimiter for SQL, as
> > defined by the ANSI standard. Double quotes are, depending on the setting
> > of some option, still supported in SQL Server 2000 for backward
> > compatibility)
> > Best, Hugo|||Allan (proflicker@.hotmail.com) writes:
> Thanks for the quick reply...
> I had already tried that and I get an error message saying:
> JOIN expression not supported
It is at this time I find it appropriate to ask which version of SQL
Server you are using. Or rather which DBMS you are using. That message
does not sound familliar at all. But it could also be that you are issueing
the query through some unknown tool which has its own quirks. Did you
try running in Query Analyzer?
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||>>Let's say I have 2 tables, a Products table and a Colors table that
go as follow: <<
1) Please post DDL, so that people do not have to guess what the keys,
constraints, Declarative Referential Integrity, datatypes, etc. in
your schema are. Sample data is also a good idea, along with clear
specifications.
2) Next, can you explain why color is NOT AN ATTRIBUTE of a product??
In your reality and therefore, your data model, it floats around with
weight, height, or whatever physcial attributes that go with being a
color?
If you had done this right, wouldn't it look more like this?
CREATE Table Products
(sku INTEGER NOT NULL PRIMARY KEY,
description VARCHAR(25) NOT NULL,
color CHAR(5) NOT NULL
CHECK(color IN (..)),
..);
>> I want to find out all the products that come in Blue, and if not I
want to have the color empty. <<
Weird, if your inventory is of any size at all; How many non-blue
things do you think that J.C. Penney's has, as compared to blue
things? But you can try this:
SELECT I1.sku, 'Blue'
FROM Inventory AS I1
WHERE color = 'Blue'
UNION ALL
SELECT I1.sku, 'Not Blue'
FROM Inventory AS I1
WHERE color <> 'Blue';
If the Colors table were actually not an attribute in a properly
designed schema, then you'd use an outer join.
Hi All,
Create a table , and put in some row 子ど and 俱
and fire a query something like SELECT * FROM mytable WHERE (myColumn = '子ど')
it gives me right result.
But if i fire
SELECT * FROM mytable WHERE (myColumn = '俱')
It does not return any result for the same even if myColumn has '俱'.
Surprisnly if i fire query like "SELECT * FROM mytable" it
correctly dispalys 俱.
What's the reason for the same ? Why does it is not able to search me on this japanese character(俱).Collation is Japanese_Unicode_CI_AS
Regards,
Sunil
Hi Sunil,you wil have to indicate that the string is unicode in your query:
SELECT * FROM mytable WHERE (myColumn = N'俱')
HTH, Jens K. Suessmeyer.
http://www.sqlserver2005.de
I have one query why it was working without N in 子ど but not in 俱
Regards,
Sunil
Hey. I need to substitute a value from a table if the input var is null. This is fine if the value coming from table is not null. But, it the table value is also null, it doesn't work. The problem I'm getting is in the isnull line which is in Dark green color because @.inFileVersion is set to null explicitly and when the isnull function evaluates, value returned from DR.FileVersion is also null which is correct. I want the null=null to return true which is why i set ansi_nulls off. But it doesn't return anything. And the select statement should return something but in my case it returns null. If I comment the isnull statements in the where clause, everything works fine. Please tell me what am I doing wrong. Is it possible to do this without setting the ansi_nulls to off? Thank you
set ansi_nulls off
go
declare
@.inFileName VARCHAR (100),
@.inFileSize INT,
@.Id int,
@.inlanguageid INT,
@.inFileVersion VARCHAR (100),
@.ExeState int
set @.inFileName = 'A0006337.EXE' set @.inFileSize = 28796 set @.Id= 1 set @.inlanguageid =null set @.inFileVersion =NULL set @.ExeState =0select Dr.StateID from table1 dR
where
DR.[FileName] = @.inFileName
AND DR.FileSize =@.inFileSizeAND DR.FileVersion = isnull(@.inFileVersion,DR.FileVersion)
AND DR.languageid = isnull(@.inlanguageid,null) AND DR.[ID]= @.ID )go
set ansi_nulls on
well actually you dont need to change the setting
if you're up to something like this
AND isnull (DR.FileVersion,-1) = isnull(@.inFileVersion,-1)
|||There is a slight problem with this. If the right side is null, it will evaluate to -1. If the left side is not null, it will evaluate to value stored in the table. It's VERY likely that the value in the table won't be -1. So the condition will be false. But, in actuality, it should be true, correct? Shouldn't it be like this?
AND isnull (DR.FileVersion,-1) = isnull(@.inFileVersion,isnull(DR.FileVersion,-1))
Thank you
|||with this
AND isnull (DR.FileVersion,-1) = isnull(@.inFileVersion,-1)
the ending equation would be
and (-1 = -1) which evaluates to true.
meaning null=null
remember that this equation resides in the "where clause" and not on the
select clause. if you want to have it returned you must
place a "case clause" in the select statement to evaluate this
nevertheless this clause must still exist in the where clause
to include the nulls
I have a table with 2 columns - "memberid" is a guid and "Interest" is a string. In an aspx page I want to read in all values of "Interest" for a given "memberid" and display approprite checkboxes as checked. The user can then change which boxes are checked and I want to record the new list of selected items in the table. I created a SQLDataSource (see below) but when I run it, I get this error:
========== Error ===========
Disallowed implicit conversion from data type sql_variant to data type uniqueidentifier, table 'DB_136571.dbo.gs_MemberInterests', column 'memberid'. Use the CONVERT function to run this query.
============================
The only use of this SQLDataSource is to delete all entries for a given user and then insert an entry for each checked checkbox. The code to set the session variables is as follows:
======== Code ============
Dim guidMemberidAs Guid =CType(user.ProviderUserKey, Guid)
Session("currmember") = guidMemberid
....
Session("currinterest") =CType(item.FindControl("CheckBox1"), CheckBox).Text
==========================
Any ideas of what I am doing wrong here?
========= SQLDataSource ==========
<asp:SqlDataSourceID="SqlDataSource2"runat="server"ConnectionString="<%$ ConnectionStrings:GoodSamSiteDB %>"
DeleteCommand="DELETE FROM [gs_MemberInterests] WHERE [memberid] = @.memberid"
InsertCommand="INSERT INTO [gs_MemberInterests] ([memberid], [Interest]) VALUES (@.memberid, @.Interest)"
SelectCommand="SELECT memberid, Interest FROM gs_MemberInterests WHERE (memberid = @.memberid) AND (Interest = @.interest)">
<DeleteParameters>
<asp:ParameterName="memberid"Type="Object"/>
</DeleteParameters>
<SelectParameters>
<asp:SessionParameterName="memberid"SessionField="currmember"Type="Object"/>
<asp:SessionParameterName="interest"SessionField="currinterest"/>
</SelectParameters>
<InsertParameters>
<asp:ParameterName="memberid"Type="Object"/>
<asp:ParameterName="Interest"Type="String"/>
</InsertParameters>
</asp:SqlDataSource>
=============================
Type="Object"|||
Motley wrote:
Type="Object"
Thanks. I assume you are referring to the fact that the "Type='Object'" has to be removed. I just found a reference to that bug. That cleared the problem.
sqlI have a SQL Server 2000 linked to a SQL Server 2005 and I am attempting populate a temporary table on the 2000 server using INSERT INTO by executing a stored procedure on the 2005 server. If I just execute the stored procedure, the data is returened with no issues. If I try to Insert the data into the temporary table the process hangs and times out.
This returns the data with no issues:
EXEC @.RetVal = MyLink.MyTable.dbo.spGetValue @.string1
This hangs and has to be killed or times out:
CREATE TABLE #TempTable (Value DECIMAL(19, 10) NULL)
INSERT INTO #TempTable
EXEC @.RetVal = MyLink.MyTable.dbo.spGetValue @.string1
SELECT * FROM #TempTable
I tried adding SET REMOTE_PROC_TRANSACTIONS OFF as suggested in an earlier post, but this had no effect.
You need to turn the MSDTC service on. You can this by clicking START > SETTINGS > CONTROL PANEL > ADMINISTRATIVE TOOLS > SERVICES. Find the service called 'Distributed Transaction Coordinator' and RIGHT CLICK (on it and select) > Start.
|||Thanks for your reply.
MSDTC service is already running on both servers.
Hello MSDN
I am using SQL 2005 and trying to INSTERT data in to a table
When I am using my command from SQL query windows it works fine,
INSERT INTO "tbl.FTPuploads" ("FTPFile_Names", "FTPGS", "FTPST", "FTPJOB", "FTPDN", "FTPSTATUS", "FTPDATE", "FTPTIME")
SELECT "FTPFile_Names", "FTPGS", "FTPST", "FTPJOB", "FTPDN", "FTPSTATUS", "FTPDATE", "FTPTIME"
FROM OpenDataSource( 'Microsoft.Jet.OLEDB.4.0',
'Data Source="G:\DATA\EDItoDB";Extended properties=Text')...uploaded#txt
But when I am trying to put that command in to a scheduled job I get this error
Executed as user: GW\fmnlasa. Incorrect syntax near 'tbl.FTPuploads'. [SQLSTATE 42000] (Error 102).The step failed.
I have changed the command to this, I have removed the quotes from the table name.
INSERT INTO tbl.FTPuploads ("FTPFile_Names", "FTPGS", "FTPST", "FTPJOB", "FTPDN", "FTPSTATUS", "FTPDATE", "FTPTIME")
SELECT "FTPFile_Names", "FTPGS", "FTPST", "FTPJOB", "FTPDN", "FTPSTATUS", "FTPDATE", "FTPTIME"
FROM OpenDataSource( 'Microsoft.Jet.OLEDB.4.0',
'Data Source="G:\DATA\EDItoDB";Extended properties=Text')...uploaded#txt
And now I get this error
Executed as user: GW\fmnlasa. Access to the remote server is denied because the current security context is not trusted. [SQLSTATE 42000] (Error 15274).The step failed.
When you execute from the query, it will use your credential to connect the remote server. But the scheduled query might use the ServiceAccount. Check which service account used in the current context & give the required permission on the remote server. Or change the service account context to NT Credential.|||Bascially the issues is of Security Context under which the Job run. Check these links
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1012759&SiteID=1
http://www.sqlmonster.com/Uwe/Forum.aspx/sql-server/44111/Access-to-the-remote-server-is-denied-because-the-current-security
Madhu
|||Thank you, for such a quick reply
I was using the same account in Job to as I was using in Query window
I am not sure but I think that the problem was in my table and column names
I have created new table with the name tblFTPuploads the old one was tbl.FTPuploads
After that I got erros for may column names so I have changed the first column name to FTPFileNames the old one was FTPfile_name
And the last step I did was to remove the quotes from the table and column names.
So the end script is like this
INSERT INTO tblFTPuploads (FTPFileNames, FTPGS, FTPST, FTPJOB, FTPDN, FTPSTATUS, FTPDATE, FTPTIME)
SELECT FTPFileNames, FTPGS, FTPST, FTPJOB, FTPDN, FTPSTATUS, FTPDATE, FTPTIME
FROM OpenDataSource( 'Microsoft.Jet.OLEDB.4.0',
'Data Source="G:\DATA\EDItoDB";Extended properties=Text')...uploaded#txt
And it is working now no problems.
Thanks again for a quick reply
Best regards,
Artavazd ASLANYAN
Network Administrator