EWF RAM, Compact Flash and log files

[Guest]

I've been experimenting with XPe on Compact Flash protected with EWF RAM.
Everything works, and I've been optimizing the image to reduce footprint.
The log files and file locations for many system apps are on the default
system drive, hence they are all on the protected CF. I noticed that event
logs are lost on a reboot of the system because of the EWF. This then made
me think that there are many other similar issues to resolve on other
applications because although the CF is protected, the log files effectively
consume RAM as the system runs. Limiting the log file size is one approach
to the problem or alternatively, changing the file location to a
non-protected drive.

Assuming that a non-protected drive is available.

What are all of the changes required to effectively run the EWF RAM
protected CF system without consuming RAM as a "normal" consequence of
running?

What files and registry setting changes are necessary? For example, the
default dump file location is %SystemRoot%\MEMORY.DMP. This information
would be lost on a crash dump unless saved to the non-protected drive.
Similarly, Shared Documents should also probably be relocated away from the
default location on the EWF RAM drive. Going further, IIS Web Server and FTP
Server, ODBC and MSDE trace and log files, etc.

Has anyone complied a list of things to do to optimize XPe when running on a
EWF RAM protected CF. (assuming that a second non-RAM protected drive is
available)

What are the implications if this second drive is another CF? Is this
advisable? Is it possible to build a CF only based XPe system that is "fully
functional" without a hard drive? By this I mean, not having to worry about
the limited number of read-write cycles of the CF relative to the hard drive.

Looking forward to your thoughts and discussion on these issues.

Joe

 

[Guest]

Clarification: I am referring to EWF RAM Reg mode as opposed to EWF RAM with
overlay.

 

[Guest]

PS: I have already done the suggestions listed at

http://msdn.microsoft.com/library/d...lp/html/xeconewfperformanceconsiderations.asp

I'm referring possibly to others such as the dump file, etc.

 

[KM]

Joe,

...
Assuming that a non-protected drive is available.

What are all of the changes required to effectively run the EWF RAM
protected CF system without consuming RAM as a "normal" consequence of
running?
http://msdn.microsoft.com/embedded/community/community/tips/xp/ramewf/default.aspx

What files and registry setting changes are necessary? For example, the
default dump file location is %SystemRoot%\MEMORY.DMP. This information
would be lost on a crash dump unless saved to the non-protected drive.

Crash dump routine bypasses EWF.

Similarly, Shared Documents should also probably be relocated away from the

Shared Documents? Can you be more specific here? If it is a system profile folder, you can change the profile locations or location
of some of profile folders.

default location on the EWF RAM drive. Going further, IIS Web Server and FTP
Server, ODBC and MSDE trace and log files, etc.

This is per application case but in general each of the app mentioned above has the path to log(s) set up in registry.

Has anyone complied a list of things to do to optimize XPe when running on a
EWF RAM protected CF. (assuming that a second non-RAM protected drive is
available)

All you've got is:
- XPe docs
- The tip page mentioned above
- This NG archive (most f the tricks were mentioned here)

What are the implications if this second drive is another CF?

None if you have the system running from CF.

Is this advisable? Is it possible to build a CF only based XPe system that is "fully
functional" without a hard drive? By this I mean, not having to worry about
the limited number of read-write cycles of the CF relative to the hard drive.

This is why you have EWF - to limit (avoid) the writings to CF.
Or are you asking about something else here?

 

[Guest]

KM, thanks for your response. I've added some other comments below.

Regards,
Joe

Joe,


http://msdn.microsoft.com/embedded/community/community/tips/xp/ramewf/default.aspx

***************
I was thinking that this list only refers to "some of the methods" to reduce
the number of writes. It did not include for example, the History folder(s)
for I.E.
Also mention of: Performance logs, remove files immediately for the recycle
bin ?

Crash dump routine bypasses EWF.


Shared Documents? Can you be more specific here? If it is a system profile folder, you can change the profile locations or location
of some of profile folders.

****************
Default Shared Documents Folder if standalone and not on a domain.
How do you change this default location to the non-protected drive?
Also, what about Folder Redirection for Documents for each users?
This is tied to the %USERPROFILE% which defaults to the C: drive.

This is per application case but in general each of the app mentioned above has the path to log(s) set up in registry.

****************
MSMQ?
I was referring to what are the steps / things to consider / required to
change the default content location, not just the log(s) for IIS FTP Server,
etc.
I know that it's per application, but ...

All you've got is:
- XPe docs
- The tip page mentioned above
- This NG archive (most f the tricks were mentioned here)


None if you have the system running from CF.


This is why you have EWF - to limit (avoid) the writings to CF.
Or are you asking about something else here?

**************
I'm thinking that if we really need a non-protected drive, and the drive is
used to save all of the logs, temporary folders, etc...
and we needed to redirect this activity away from the CF primary system drive,
does this imply that the non-protected drive must therefore be capable of
handling the high volume of read-write cycles re-directed away from the CF,
hence can the non-protected drive also be a CF?
Question being -
are we protecting the CF from the Windows OS itself using the EWF RAM Reg,
and the redirected items to the non-protected drive are required mainly for
persistence?
or are these the high frequency read-write cycle items that we are
protecting the CF from?
Secondly, if the second drive is also a CF, then there is still no ability
to use this as
the paging file, hence there may be a strong argument that somehow, a HDD is
still recommended.

Our application is a data acquisition system which is 24 / 7 / 365 operation
application.
We would like to deploy without a hard disk drive for robustness and
vibration tolerance / immunity.
Acquisition rate / data saved to database is approx. once every 5 seconds,
approx. 200 records.
Considering OS and applications on the first CF and database on a second CF.
MSDE / SQL Server limitation of 2 - 4 GB on second CF is okay.
Would like to implement Terminal Services, and IIS Technologies - hence IIS
Server and FTP Server.
Automatic daily archiving of data (incremental) to FTP server directory for
backup on network drive.

Can / should this be done without a HDD?
or should I simply consider adding a HDD, add the page file, etc...
and not worry about anything related to the OS, RAM consumption, log files,
etc?
but then again, introducing the HDD rotating media implies that maybe I
can't have the
advantages of a completely CF based solution, hence I still need to worry,
so why use a CF at all?

I realize that there may be some applications that may be suitable for CF
only deployment, but maybe mine is not...

 

[KM]

Joe,

All your question do make perfect sense since you are deploying image on CF.

However, you may also know that although XPe is componentized version of XP Pro pretty much none of the applications there (except
EEFs) were developed with embedded application in mind.

The real challenge in XPe has always been that all of the MS apps are provided in binary form and no sources available. This makes
it sometimes extremely hard to customize Microsoft applications for using in a particular embedded scenario. Personally I think it
is a very weak side of XPe compared to even other Microsoft embedded OSes like Windows CE. Btw, have you thought of using Windows CE
for your device?

In your case, you don't just want to redirect application specific logs and data to an unprotected volume (some registry hacks were
posted in this NG for some application from your list). But you also want to limit the logs or, even better using with EWF in place,
make sure that the file space for log or data files is pre-allocated so that you don't need to worry about EWF growing overlay size.
It is obvious that applications like MSMQ, Event Log and etc. are definitely missing the last feature as they have been developed
for XP Pro.

So, in reality if you image is fat and include heavy software like IIS, MSMQ, MSDE, etc. nothing will safe you from the growing EWF
overlay size unless you move or the writes to unprotected partition. But then you are probably stuck with HDD there.
Another way may be using expensive flash solutions like M-Systems uDoc. These may be used as almost like a replacement to HDD from
software point of view. But hardware wise are robust and vibration tolerant and etc.

To cover all the tweaks for particular software components I'd recommend you to create a thread per each component so that people
who know the tweaks can reply. As far as I know there is no available FAQ fro XPe which lists all the tweaks you need and mentioned
below. Although this NG archive will help you to cover some [most] of them.

 

[Guest]

KM,

Thanks for your comments and for confirming some of my suspicions. I really
appreciate it that you and SB always take the time to respond to the numerous
questions by so many people. I try to pitch in whenever I can. I will
continue to evaluate the various components and options to determine which
ones can be configured / tweaked and will post my findings in the future.

Regards,
Joe

Joe,

All your question do make perfect sense since you are deploying image on CF.

However, you may also know that although XPe is componentized version of XP Pro pretty much none of the applications there (except
EEFs) were developed with embedded application in mind.

The real challenge in XPe has always been that all of the MS apps are provided in binary form and no sources available. This makes
it sometimes extremely hard to customize Microsoft applications for using in a particular embedded scenario. Personally I think it
is a very weak side of XPe compared to even other Microsoft embedded OSes like Windows CE. Btw, have you thought of using Windows CE
for your device?

In your case, you don't just want to redirect application specific logs and data to an unprotected volume (some registry hacks were
posted in this NG for some application from your list). But you also want to limit the logs or, even better using with EWF in place,
make sure that the file space for log or data files is pre-allocated so that you don't need to worry about EWF growing overlay size.
It is obvious that applications like MSMQ, Event Log and etc. are definitely missing the last feature as they have been developed
for XP Pro.

So, in reality if you image is fat and include heavy software like IIS, MSMQ, MSDE, etc. nothing will safe you from the growing EWF
overlay size unless you move or the writes to unprotected partition. But then you are probably stuck with HDD there.
Another way may be using expensive flash solutions like M-Systems uDoc. These may be used as almost like a replacement to HDD from
software point of view. But hardware wise are robust and vibration tolerant and etc.

To cover all the tweaks for particular software components I'd recommend you to create a thread per each component so that people
who know the tweaks can reply. As far as I know there is no available FAQ fro XPe which lists all the tweaks you need and mentioned
below. Although this NG archive will help you to cover some [most] of them.

--
Regards,
KM

KM, thanks for your response. I've added some other comments below.

Regards,
Joe


***************
I was thinking that this list only refers to "some of the methods" to reduce
the number of writes. It did not include for example, the History folder(s)
for I.E.
Also mention of: Performance logs, remove files immediately for the recycle
bin ?

****************
Default Shared Documents Folder if standalone and not on a domain.
How do you change this default location to the non-protected drive?
Also, what about Folder Redirection for Documents for each users?
This is tied to the %USERPROFILE% which defaults to the C: drive.

****************
MSMQ?
I was referring to what are the steps / things to consider / required to
change the default content location, not just the log(s) for IIS FTP Server,
etc.
I know that it's per application, but ...

**************
I'm thinking that if we really need a non-protected drive, and the drive is
used to save all of the logs, temporary folders, etc...
and we needed to redirect this activity away from the CF primary system drive,
does this imply that the non-protected drive must therefore be capable of
handling the high volume of read-write cycles re-directed away from the CF,
hence can the non-protected drive also be a CF?
Question being -
are we protecting the CF from the Windows OS itself using the EWF RAM Reg,
and the redirected items to the non-protected drive are required mainly for
persistence?
or are these the high frequency read-write cycle items that we are
protecting the CF from?
Secondly, if the second drive is also a CF, then there is still no ability
to use this as
the paging file, hence there may be a strong argument that somehow, a HDD is
still recommended.

Our application is a data acquisition system which is 24 / 7 / 365 operation
application.
We would like to deploy without a hard disk drive for robustness and
vibration tolerance / immunity.
Acquisition rate / data saved to database is approx. once every 5 seconds,
approx. 200 records.
Considering OS and applications on the first CF and database on a second CF.
MSDE / SQL Server limitation of 2 - 4 GB on second CF is okay.
Would like to implement Terminal Services, and IIS Technologies - hence IIS
Server and FTP Server.
Automatic daily archiving of data (incremental) to FTP server directory for
backup on network drive.

Can / should this be done without a HDD?
or should I simply consider adding a HDD, add the page file, etc...
and not worry about anything related to the OS, RAM consumption, log files,
etc?
but then again, introducing the HDD rotating media implies that maybe I
can't have the
advantages of a completely CF based solution, hence I still need to worry,
so why use a CF at all?

I realize that there may be some applications that may be suitable for CF
only deployment, but maybe mine is not...