Category Archives: Programming & Development
SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in position truncated \UXXXXXXXX escape
This is a common error in python when you are reading from a file and have specified a file path String. You can usually fix this by placing an ‘r’ in the front of your string to change it to a raw string.

Before fix (code has the issue)
book = epub.read_epub("C:\Users\ronni\Desktop\ebook\Frankenstein.epub")
After fix (issue is resolved)
book = epub.read_epub(r"C:\Users\ronni\Desktop\ebook\Frankenstein.epub")
Explanation
Python interprets “\” within a string as a special escape character. In this example, the \U in position 2-3 is a unicode escape character. Adding ‘r’ instructs the python interpreter to instead treat the string as a raw literal.
References
https://blog.softhints.com/python/
heroku cheat sheet quick start
#note requires java 8. add the ppa repo for oracle and install this if you dont have it already. check your local with: java -version
#create auth token for subsequent commands
heroku login
#from within a local git cloned app root, to create a new dyno
heroku create
#deploy
git push heroku master
#check if dyno is running
heroku ps:scale web=1
#shortcut to open the dyno. if you are running these in a remote SSH you will need to X11 forward for this to work
#or simply, just open the address indicated after you push
#this uri is also visible using the logs command below
heroku open
#tail logs for running dyno. refresh the browser while you are viewing the app to verify new log entries added
heroku logs --tail
#procfile declares what command is executed at start of app. this is likely web.
#for windows: this is different. see heroku docs and make Procfile.windows and use heroku -f Procfile.windows
https://devcenter.heroku.com/articles/procfile
#general note regarding the “free-ness” of heroku on your first app
“By default, your app is deployed on a free dyno. Free dynos will sleep after a half hour of inactivity and they can be active (receiving traffic) for no more than 18 hours a day before going to sleep. If a free dyno is sleeping, and it hasn’t exceeded the 18 hours, any web request will wake it. This causes a delay of a few seconds for the first request upon waking. Subsequent requests will perform normally”
#pom.xml specifies dependencies
#system.properties in the case of java determines version of java to use
#pull in dependencies to target/dependency and store this in pom.xml
mvn clean install
#to start heroku locally
heroku local web
#install add-ons
heroku addons:create papertrail
#list add-ons
heroku addons
#open add-on logs
heroku addons:open papertrail
#test a command on a one-off dyno, e.g. run bash in local dyno
heroku run bash
#set environment variables
heroku config:set ENERGY="20 GeV"
#view environment variables
heroku config
#show database info, e.g. postgres
heroku pg
#run tsql commands
heroku pg:psql
References:
https://devcenter.heroku.com/articles/getting-started-with-java
javac invalid target release maven ant intellij netbeans eclipse
You may see the following error when running maven or ant command or from intellij:
javac: invalid target release: 1.8
This occurs because you are running a build target for a version of java that you do not have set as your default in JAVA_HOME.
If you do not have java 8 installed, in ubuntu this is as simple as:
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
This is similar in other flavors of linux, simply add the official oracle repo and install. Installation in Windows requires browsing to the JDK/JRE pages on Oracle and downloading/installing. Avoid OpenJDK. This is outside the context of this blog post, but you can google “openjdk vs oracle jdk performance” to learn more.
If you see this error in an IDE such as intellij, netbeans or eclipse, this can be fixed by setting the java compile in your Settings to Java 8. E.g. if the error above said 1.9, change this to Java 9.
If you are upgrading from 7 to 8, the last oracle install step when installing the package should set JAVA_HOME:
(ubuntu) sudo apt-get install oracle-java8-set-default
If the above install command or corresponding install script on your system does not work, to fix you can also manually set JAVA_HOME, such as:
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
echo the current $JAVA_HOME and the new oracle java home should be alongside it.
In my scenario I actually ran the oracle provided install script but it failed to set the java home correctly so I still had to do this manually.
http://tecadmin.net/install-oracle-java-8-jdk-8-ubuntu-via-ppa/#
solarized color theme
Parent site:
http://ethanschoonover.com/solarized
Vim (standalone or with pathogen):
https://github.com/altercation/vim-colors-solarized
Visual Studio (also in main repo):
https://github.com/leddt/visualstudio-colors-solarized
IntelliJ (also in main repo):
https://github.com/jkaving/intellij-colors-solarized
Putty:
Install the .reg files to use with new sessions or edit previous sessions following the provided instructions.
https://github.com/brantb/solarized/tree/master/putty-colors-solarized
After you are connected to remote machine, you may need to run:
export TERM=xterm-256color
vimrc settings
Located in ~/.vimrc
If it doesn’t exist, create it!
set smartindent set tabstop=2 set shiftwidth=2 set expandtab "enables mouse in all modes. supports click and scrollwheel. use shift+ins to paste from clipboard set mouse=a "for use with pathogen syntax enable filetype plugin indent on "for use with solarized. may require export TERM=xterm-256color set background=dark colorscheme solarized
To see more info on these settings, from within vim:
:help smartindent
To format a file with mixed tabs and spaces, after applying the above settings, from within vim:
:%retab
Query Sql Version Microsoft MSSQL
set nocount on go select @@version go select 'Edition: ' + convert(char(30), serverproperty('Edition')) go select 'Product Version: ' + convert(char(20), serverproperty('ProductVersion')) go select 'Product Level: ' + convert(char(20),serverproperty('ProductLevel')) go set nocount off go
References
https://support.microsoft.com/kb/321185
http://blogs.msdn.com/b/euanga/archive/2008/01/23/confused-by-sql-server-version-numbers.aspx
ASP .NET Code Contracts Examples
//contract.requires public void Withdraw(Account account, decimal amount) { Contract.Requires(account != null, "account cannot be null"); Contract.Requires(amount > 0, "amount must be greater than zero"); ... } //contract.ensures public double CalculateDiscount(int originalAmount, string discountCode) { Contract.Ensures(Contract.Result<double>() >= 0); return -32; }
Reference
http://visualstudiomagazine.com/articles/2010/06/23/code-contracts.aspx
http://msdn.microsoft.com/en-us/library/dd264808(v=vs.110).aspx
TPL Task Parrallel Library .net 4.0 parallel programming multi thread threading task queue user work item
//simple parallel function call Parallel.Invoke(() => somefunc1(), () => somefunc2()); //parallel foreach with two lists new List<List> tasks = new List<List>() { new List() { new DataSource1ProcessorA() }, new List() { new DataSource2ProcessorA(), new DataSource2ProcessorB() } } Parallel.ForEach(tasks, (items) => { foreach (var item in items) { item.GetData(); item.Process(); } }); //new task static factory method approach Task.Factory.StartNew(() => { data.GetData(); data.Process(); }) .ContinueWith(t => Logger.Error("An exception occurred while processing. Check the inner exception for details", t.Exception), TaskContinuationOptions.OnlyOnFaulted); //explicit object oriented // Create a task and supply a user delegate by using a lambda expression. Task taskA = new Task( () => Console.WriteLine("Hello from taskA.")); // Start the task. taskA.Start(); // Output a message from the calling thread. Console.WriteLine("Hello from thread '{0}'.", Thread.CurrentThread.Name); taskA.Wait(); //using task run Thread.CurrentThread.Name = "Main"; // Define and run the task. Task taskA = Task.Run( () => Console.WriteLine("Hello from taskA.")); // Output a message from the calling thread. Console.WriteLine("Hello from thread '{0}'.", Thread.CurrentThread.Name); taskA.Wait();
References
http://msdn.microsoft.com/en-us/library/dd460705.aspx
(added 20121023 good comprehensive resource with illustrative examples)
http://www.codeproject.com/Articles/362996/Multi-core-programming-using-Task-Parallel-Library
http://www.codeproject.com/KB/threads/ParallelTasks.aspx
Optimize Managed Code For Multi-Core Machines
http://msdn.microsoft.com/en-us/magazine/cc163340.aspx
good blog
http://www.codethinked.com/net-40-and-systemthreadingtasks
queueuserworkitem
http://msdn.microsoft.com/en-us/library/4yd16hza.aspx
threadpool class
http://msdn.microsoft.com/en-us/library/y5htx827.aspx
task class
http://msdn.microsoft.com/en-us/library/system.threading.tasks.task.aspx
task parrallelism
http://msdn.microsoft.com/en-us/library/dd537609.aspx
How to: Use Parallel.Invoke to Execute Parallel Operations
http://msdn.microsoft.com/en-us/library/dd460705.aspx
Custom DAL Class SQL ORM ASP .NET
(common.DataObject may be of your choosing or may simply replace with dynamic)
using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Data; using System.Web.UI.WebControls; using System.Web.UI; using System.Data.SqlClient; using AIS.Common; //this is a common namespace I use in examples throughout my site using System.Reflection; using System.Dynamic; //TODO: consider returning ienumerable in sp return values for lazy eval vs .tolist immediate eval namespace AIS.DAL.AppName { public static class StoredProcedures { public delegate void ErrorHandler(Exception ex); /// <summary> /// If no custom error handling is bound to this event, exceptions will be thrown back up to the calling function. /// If custom handling is bound to this event, ensure it does not perform a redirect or kill the thread unless you intend to abort the procedural /// steps following the method/function call which threw the error. /// </summary> public static event ErrorHandler HandleError; #region Unique Procedures public static List<Common.DataObject> LoadUserSessions_All(dynamic o) { return ExecuteRead("an_get_db_fn1", o); } public static List<Common.DataObject> LoadUserSessionsDetails_LiveStream(dynamic o) { return ExecuteRead("an_get_db_fn2", o); } public static List<Common.DataObject> LoadUserSessionsDetails_Live(dynamic o) { return ExecuteRead("an_get_db_fn3", o); } public static int LogChat() { return ExecuteScalar("an_get_db_fn4", null); } public static int LogError() { return ExecuteScalar("an_get_db_fn5", null); } #endregion //TODO: consider hiding from external assemblies which would require strong mappings above #region Execution Logic public static List<Common.DataObject> ExecuteRead(string procedurename, dynamic param) { try { SqlDataSource sds = new SqlDataSource(); sds.ConnectionString = ConfigValues.TrainingPortalConnectionString; sds.SelectCommandType = SqlDataSourceCommandType.StoredProcedure; sds.SelectCommand = procedurename; if (param != null) { foreach (PropertyInfo pi in param.GetType().GetProperties()) { object pval = pi.GetValue(param, null); if (pval != null) { sds.SelectParameters.Add(pi.Name, pval.ToString()); } } } List<Common.DataObject> results = new List<Common.DataObject>(); //sds.Select(new DataSourceSelectArguments()).Cast<DataRowView>().ToList().ForEach(o => Load_AddResult<dynamic>(o, ref results)); sds.Select(new DataSourceSelectArguments()).Cast<DataRowView>().ToList().ForEach(o => Load_AddResult<Common.DataObject>(o, ref results)); return results; } catch (Exception ex) { HandleError_Condensed(ex); return null; } } public static void Load_AddResult<t>(Common.DataObject o, ref List<t> results) { try { t r = (t)Activator.CreateInstance(typeof(t)); foreach (PropertyInfo pi in typeof(t).GetProperties()) { object v = o[pi.Name].ToString(); Type pt = Type.GetType(pi.PropertyType.FullName); //try { pi.SetValue(r, Convert.ChangeType(v, pt), null); } //catch (Exception ex) { HandleError_Condensed(ex); } o.Add(pi.Name, Convert.ChangeType(v, pt)); } results.Add(r); } catch (Exception ex) { HandleError_Condensed(ex); } } //public static void Load_AddResult<t>(dynamic o, ref List<t> results) //{ // try // { // t r = (t)Activator.CreateInstance(typeof(t)); // foreach (PropertyInfo pi in typeof(t).GetProperties()) // { // object v = o[pi.Name].ToString(); // Type pt = Type.GetType(pi.PropertyType.FullName); // try { pi.SetValue(r, Convert.ChangeType(v, pt), null); } // catch (Exception ex) { HandleError_Condensed(ex); } // } // results.Add(r); // } // catch (Exception ex) // { // HandleError_Condensed(ex); // } //} public static void ExecuteNonScalar(string procedurename, dynamic param) { try { ExecuteScalar(procedurename, param); } catch (Exception ex) { HandleError_Condensed(ex); } } public static int ExecuteScalar(string procedurename, dynamic param) { try { SqlDataSource sds = new SqlDataSource(); sds.ConnectionString = ConfigValues.TrainingPortalConnectionString; sds.UpdateCommandType = SqlDataSourceCommandType.StoredProcedure; sds.UpdateCommand = procedurename; if (param != null) { foreach (PropertyInfo pi in param.GetType().GetProperties()) { object pval = pi.GetValue(param, null); if (pval != null) { sds.SelectParameters.Add(pi.Name, pval.ToString()); } } } return sds.Update(); } catch (Exception ex) { HandleError_Condensed(ex); return 1; //1 signifies error in tsql } } #endregion private static void HandleError_Condensed(Exception ex) { if (HandleError != null) { HandleError(ex); } else { throw new Exception(ex.Message, ex); } } } }
Custom AWS S3 Helper Class AWSSDK Wrapper
Implements some very commonly used AWS S3 functionality. (need to merge with my other AWS wrapper classes, Route53, EC2, etc)
using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.IO; using System.Collections.Specialized; using System.Configuration; //uses AWSSDK.dll from amazon using Amazon; using Amazon.S3; using Amazon.S3.Model; using System.Xml.Linq; using System.Xml; using System.Data; namespace AIS.Common { public static class AWSHelper { private static List<S3Bucket> LoadS3Buckets() { System.Collections.Specialized.NameValueCollection appConfig = System.Configuration.ConfigurationManager.AppSettings; using (var s3client = Amazon.AWSClientFactory.CreateAmazonS3Client(ConfigValues.AWSAccessKey, ConfigValues.AWSSecretKey)) { return s3client.ListBuckets().Buckets; } } private static List<S3Object> LoadS3Objects(string bucketname) { System.Collections.Specialized.NameValueCollection appConfig = System.Configuration.ConfigurationManager.AppSettings; using (var s3client = Amazon.AWSClientFactory.CreateAmazonS3Client(ConfigValues.AWSAccessKey, ConfigValues.AWSSecretKey)) { return s3client.ListObjects(new ListObjectsRequest() { BucketName = bucketname }).S3Objects; } } private static void LoadS3File(string bucketname, string keyname, HttpResponse response, string contenttype) { NameValueCollection appConfig = ConfigurationManager.AppSettings; using (var s3client = Amazon.AWSClientFactory.CreateAmazonS3Client(ConfigValues.AWSAccessKey, ConfigValues.AWSSecretKey)) { GetObjectRequest s3request = new GetObjectRequest() .WithBucketName(bucketname).WithKey(keyname); using (GetObjectResponse s3response = s3client.GetObject(s3request)) { string title = s3response.Metadata["x-amz-meta-title"]; response.Clear(); //Response.Write(string.Format("The object's title is {0}", title)); //Response.AddHeader //Response.ContentType="application/swf"; ////Response.ContentType="contenttype"; response.ContentType = s3response.ContentType; //s3response.Headers["Content-Length"]; long filesize = s3response.ContentLength; byte[] buffer = new byte[(int)filesize]; response.BinaryWrite(ConvertStreamToBytes(s3response.ResponseStream, filesize)); response.Flush(); response.Close(); } } } public static string GetS3UrlToVideo(string bucketname, string keyname) { System.Collections.Specialized.NameValueCollection appConfig = System.Configuration.ConfigurationManager.AppSettings; string url = ""; using (var s3client = Amazon.AWSClientFactory.CreateAmazonS3Client(ConfigValues.AWSAccessKey, ConfigValues.AWSSecretKey)) { Amazon.S3.Model.GetPreSignedUrlRequest request = new Amazon.S3.Model.GetPreSignedUrlRequest() .WithBucketName(bucketname) .WithKey(keyname) .WithProtocol(Amazon.S3.Model.Protocol.HTTP) .WithVerb(HttpVerb.GET) .WithExpires(DateTime.Now.AddMinutes(ConfigValues.VideoURLExpiration)); Amazon.S3.Model.GetPreSignedUrlRequest r = new GetPreSignedUrlRequest(); url = s3client.GetPreSignedURL(request); url= "https://s3.amazonaws.com/" + bucketname + keyname; } //return System.Xml.XmlConvert.EncodeName(url); return url; } public static byte[] ConvertStreamToBytes(Stream input, long filesize) { byte[] buffer = new byte[(int)filesize]; using (MemoryStream ms = new MemoryStream()) { int read; while ((read = input.Read(buffer, 0, buffer.Length)) > 0) { ms.Write(buffer, 0, read); } return ms.ToArray(); } } } }